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ABSTRACT 


Since  the  early  1960s,  the  Department  of  Defense  and 
the  air  transport  industry  have  seen  a  downward  trend  in  the 
number  of  new  aircraft  production  starts.  One  of  the  effects 
of  fewer  new  development  programs  has  been  a  declining  level 
of  practical  design  experience  acquired  by  individual  engi¬ 
neers  in  aerospace  design  organizations.  When  compared  to 
the  growing  need  for  design  experience  build-up,  a  result  of 
expanding  technology,  the  situation  becomes  worse.  To 
acquire  needed  levels  of  practical  design  experience,  feed¬ 
back  and  utilization  of  operational  experience  is  becoming 
increasingly  important.  Responsive  feedback  systems  are 
used  by  the  commercial  air  transport  industry  for  providing 
operational  experience  applicable  to  product  improvement  and 
new  development  programs.  Feedback  systems  in  Naval  aviation 
provide  data  and  information  for  application  primarily  in 
areas  of  manpower  and  material  management.  This  thesis 
analyzes  and  discusses  the  present  situation  and  basic  needs 
for  operational  experience  feedback  in  aircraft  design. 
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I .  INTRODUCTION 


Why  is  it  that  the  Department  of  the  Navy  frequently 
experiences  similar  or  repetitive  undesirable  design  charac¬ 
teristics  from  one  aeronautical  weapon  system  development 
program  to  the  next?  Why  is  it  that  the  many  enhancing  design 
characteristics  seen  on  currently  deployed  and  many  retired 
aircraft  are  not  exploited  during  new  development  programs? 
Questions  of  this  type,  asked  by  fleet  operating  units  and 
test  and  evaluation  activities,  have  surfaced  as  a  result  of 
the  rapidly  increasing  number  of  engineering  change  proposals, 
escalating  life  cycle  costs,  under-achieved  mission  readiness 
rates,  and  in  many  cases,  incompatibilities  with  military 
operating  environments. 

Although  the  above  questions  reflect  a  viable  problem, 
they  are  not  intended  to  imply  that  the  Navy  is  unconcerned 
about  operational  experience  applicable  to  design  of  aircraft 
weapon  systems.  On  the  contrary,  ambitious  programs  were 
undertaken  by  the  Naval  Air  Systems  Command  during  the  1970s 
on  grappling  the  many  problems  associated  with  systems 
effectiveness  parameters  applicable  to  major  system  acquisi¬ 
tion.  The  programs  emphasized  utilizing  available  operational 
experience  for  developing  specific  and  more  meaningful 
specification  requirements. 

While  considerable  effort  has  been  expended  on  developing 
specification  requirements,  little  effort  has  been  allocated 
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to  improving  operational  experience  feedback  from  fleet 
operators  to  industry  designers.  Inadequacies  in  current 
feedback  systems  inhibit  effective  utilization  of  operational 
experience  in  the  front  end  of  new  design.  Hence,  it  is  this 
notion  that  was  pursued  by  this  study. 


A.  PURPOSE  OF  THESIS 

The  purpose  of  this  thesis  is  twofold.  First,  to 
investigate,  analyze,  and  describe  the  present  situation  and 
basic  trends  regarding  feedback  of  operational  experience 
from  user  to  manufacturer  of  commercial  and  military  aircraft; 
and  second,  to  summarize  and  discuss  the  findings. 

B.  SCOPE  OF  STUDY 

Constraints  of  time  and  resources  limited  this  investiga¬ 
tion  to  the  Department  of  the  Navy  for  military  aircraft  and 
to  the  Lockheed-California  Company  for  commercial  transport 
aircraft. 

The  scope  of  this  study  is  confined  to: 

1.  Investigating  the  validity  and  need  for  feedback 
of  operational  experience; 

2.  Studying  feedback  sources  on  the  commercial  side 
of  the  aerospace  industry; 

3.  Studying  current  Naval  Aviation  feedback  sources; 
and 

4.  Determining  current  situation  for  feedback  of 
operational  experience  to  designers. 
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To  avoid  later  confusion,  it  is  imperative  that  the 
reader  understands  that  this  thesis  addresses  experience 
feedback  only.  The  equally  important  topic  of  experience 
transfer  is  not  within  the  scope  of  study.  Sub-section  E 
of  the  Introduction  and  Figure  1  briefly  differentiate  the 
two.  While  feedback  is  basic  to  operational  experience 
build-up  within  a  given  design  engineering  organization, 
transfer  provides  for  sharing  feedback  experience  with  other 
organizations  as  well  as  distributing  it  to  those  individuals 
who  have  a  "need  to  know"  inside  an  organization. 

C.  METHODOLOGY  EMPLOYED 

One  of  the  difficulties  encountered  in  the  study  was 
finding  reference  sources,  government  or  industry,  that 
addressed  the  subject  directly.  Extensive  reasearch  revealed 
little  work  has  been  published  in  this  topic  area. 

The  major  source  of  information  used  as  the  primary  basis 
for  study  was  obtained  through  extensive  personal  interviews 
conducted  by  the  author.  All  interviews  were  begun  with  a 
complete  explanation  of  the  nature  of  the  research.  The 
interviews  were  not  formalized  but  were  tailored  to  the 
interviewee  and  were  intended  to  provide  the  author  with  an 
insight  into  the  atmosphere,  attitude  and  functions  of  the 
organization  being  interviewed.  The  goal  was  to  establish 
a  rapport  with  the  interviewee  and  then  find  out  his  candid 
opinions  on  the  various  aspects  of  operational  experience 
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compilation,  feedback  and  utilization.  Organizations 
contacted  and  interviewed  during  the  course  of  the  study  are 
listed  in  Appendix  A. 

D.  GLOSSARY  OF  TERMS 

The  following  brief  descriptions  are  provided  to  clarify 
some  basic  terms  as  treated  by  this  study. 

1.  Design  has  been  defined  in  many  ways.  For  the 
present  purposes,  among  the  many  aspects  of  design, 
an  emphasis  on  learning  from  past  experience  is  of 
special  importance. 

2.  Aircraft  Design  offers  an  outstanding  example  for  the 
complex  demands  and  responsibilities  in  the  field  of 
design.  Performance,  safety,  cost,  weight,  relia¬ 
bility  and  maintainability  are  some  of  the  contra¬ 
dictory  demands  which  have  to  be  met  and  reconciled. 
The  responsibilities  of  the  designer  extend  over  the 
whole  service  life  of  the  aircraft,  with  innumerable 
combinations  of  environmental  and  operational 
conditions . 

3.  Data  are  the  original  and  detailed  representations 
of  events  in  the  physical  world.  Data  are  the  raw 
material  from  which  information  is  manufactured  by 
operations  known  collectively  as  data  processing. 

4.  Information  can  be  thought  of  as  processed  data  or 
data  given  context.  Information  is  that  knowledge 
that  can  be  applied  in  current  operations. 
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5.  Experience  is  a  special  type  of  information.  The 
word  experience  has  its  origin  in  the  Latin  "from 
having  gone  through"  —  in  contrast  to  the  word 
theory,  which  is  derived  from  the  Greek  "to  look  at." 
Experience  refers  to  having  gone  through  a  real-life 
process.  A  very  large  number  of  parameters  may  be 
involved  and  it  is  of  paramount  importance  to 
separate  essential  from  non-essential  parameters. 

6.  Operational  has  to  do  with  the  operation  or  use  of 
a  system,  sub-system,  component  in  the  environment 
and  for  the  purpose  which  it  was  intended. 

7.  Feedback,  in  the  present  context,  pertains  to  the 
flow  of  operational  data,  information,  and/or 
experience  from  the  user  back  to  the  contractor. 

8.  Transfer  pertains  to  the  dissemination  or  exchange 
of  operational  data,  information,  and/or  experience 
among  contractors,  as  shown  in  Figure  1,  or  among 
individuals  within  an  organization. 

9.  Experience  Feedback  in  Aircraft  Design  is  concerned 
with  transmitting  "lessons  learned"  in  operational 
experience  to  the  organization  responsible  for  the 
product. 

10.  Experience  Transfer  in  Aircraft  Design  is  concerned 
with  transmitting  "lessons  learned"  in  operational 
experience  to  designers  throughout  the  industry  who 
have  a  "need  to  know." 
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Figure  1  FEEDBACK  FLOW  AND  TRANSFER  SIMPLIFIED. 
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II.  BASIC  TRENDS  IN  AIRCRAFT  DESIGN 


A .  BACKGROUND 

Without  doubt,  the  best  and  most  effective  implementation 
of  operational  experience  occurred  during  the  pioneering 
years  of  powered  flight.  As  both  designer  and  user,  the 
early  pioneers,  such  as  the  Wright  Brothers  and  Glenn 
Curtiss,  were  able  to  readily  learn  from  their  previous 
designs  and  immediately  apply  the  experiences  to  their  next 
design  decision.  During  the  infant  years  of  aviation,  design 
groups  remained  small.  However,  as  aviation  grew  into  an 
industry,  and  the  technology  associated  with  flight  became 
more  involved,  the  likelihood  of  the  designer  also  being  the 
ultimate  user  became  rare.  Nonetheless,  the  customer  or 
ultimate  user  of  these  early  aircraft,  more  often  than  not, 
worked  hand-in-hand  with  the  designer  in  an  effort  to  gain 
the  best  possible  design  consistent  with  technology  and 
operational  experience  of  that  time. 

By  the  early  1930s,  designers  knew  that  the  successful 
aircraft  of  the  next  generation  would  be  more  complex  with 
retractable  landing  g’ar  and  all  the  other  emerging  refine¬ 
ments  of  variable-pitch  propellers,  wing  flaps,  control 
surface  trim  tabs,  autopilot,  de-icing  equipment  and  radio 
navigation  capabilities.  The  talent  required  to  success¬ 
fully  design  and  build  an  aircraft  of  this  complexity  was 
beyond  the  ability  of  the  " jack-of-all-trades"  designer  of 
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the  time.  Required  were  the  coordinated  inputs  from  a 
number  of  growing  aeronautical  specialists  in  such  disciplines 
as  aerodynamics,  structures,  propulsion,  hydraulics,  and 
avionics.  Hence,  the  design  organization  was  born. 

During  the  1960s,  the  aircraft  industry  sought  to  improve 
the  rapport  between  designer  and  user  by  introducing  formal 
feedback  systems  by  which  operational  experience  from  past 
aircraft  development  programs  could  be  compiled,  transferred, 
and  utilized  on  derivative  or  next  generation  development 
programs.  This  interest  largely  resulted  from  rapidly 
escalating  program  development  costs  and  the  "hard-nosed" 
position  taken  first  by  the  air  transport  industry  and  later 
by  the  Department  of  Defense  (DoD)  to  improve  life-cycle 
costs  through  improved  reliability  and  maintainability. 

B.  FEWER  NEW  DEVELOPMENT  STARTS 

A  long-term  reduction  in  the  number  of  new  development 
starts  for  DoD  began  in  the  second  half  of  the  1950s.  Trend 
data  indicates  a  steady  declining  number  of  new  aircraft 
production  starts.  The  trend  as  shown  in  Figure  2  appears 
to  be  approaching  an  asymptote  of  just  under  one  when  taking 
a  seven  year  moving  average.  This  equates  to  a  projected 
1.5  new  starts  between  1985-1995.  [18:37]^ 

*This  notation  is  used  throughout  the  report  for  sources 
of  major  reference.  The  first  number  is  the  source  listed 
in  the  List  of  References.  The  second  number  is  the  page  in 
the  source. 
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NUMBER  OF  AIRCRAFT 
STARTS 


This  downward  trend  has  had  an  impact  on  all  military 
aircraft  programs  including  fighter,  attack,  special  mission, 
transport,  utility  and  helicopter  types.  Illustrative  of 
the  trend  is  Figure  3,  which  depicts  specific  new  development 
program  starts  for  tactical  (fighter  and  attack)  aircraft. 

Of  particular  note  is  the  slowdown  in  prototype  development 
for  the  decade  between  1959  and  1969.  Only  two  new  starts 
occurred  during  the  period,  largely  a  result  of  McNamara's 
"paper  competition"  policies  that  disallowed  prototype 
development.  The  spurt  of  new  programs  begun  during  the 
early  1970s  was  most  likely  a  "catch-up"  of  programs  post¬ 
poned  or  suppressed  during  the  1960s. 

The  commercial  air  transport  industry  experienced  a 
similar  downturn  in  new  aircraft  developments.  Current 
inventory  airline  flight  equipment,  including  Boeing  B707/ 
B727/B737/B747,  McDonnell  Douglas  DC- 8/DC- 9/DC- 10 ,  and 
Lockheed  L-1011  aircraft,  are  a  product  of  the  1950s  and 
1960s.  Until  Boeing's  early  1979  announcement  to  launch 
development  of  their  next  generation  B757  and  B767  aircraft, 
the  industry  had  not  seen  a  new  development  start  for  a 
decade . 

Greater  periods  of  lapsed  time  between  military  and 
commercial  aircraft  development  programs,  a  result  of  fewer 
new  starts,  has  had  an  adverse  effect  on  retained  employment 
of  experienced  designers  in  the  industry,  commercial  and 
military  alike. 
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C.  DETERIORATING  DESIGN  TEAM  CONTINUITY 


Maintaining  design  team  continuity  is  another  of  the 
growing  concerns  to  the  aerospace  industry.  The  era  of 
fewer  new  starts  and  resultant  excess  engineering  capacity 
has  ushered  in  disenchantment  among  aircraft  designers  and 
engineering  professionals  for  long-term  career  opportunities 
in  the  industry.  Many  of  these  experienced  professionals 
have  pursued  more  promising  career  opportunities  in  another 
industry.  Others  are  "job-hopping"  to  companies  that  have 
been  successful  in  landing  contracts  for  new  development 
starts . 

Continuity  within  engineering  design  teams  is  obviously 

important  and  was  recognized  as  such  by  industry  veteran 

Mr.  Edward  H.  Heinemann  in  a  presentation  on  "New  Airplane 

Development."  Heinemann  stated: 

Successful  manufacturers  the  world  over  have 
found  continuity  of  design  and  manufacture  a 
necessity.  Furthermore,  three  to  five  projects 
in  various  stages  of  development  are  necessary 
at  all  times  to  remain  competitive  and  retain 
healthy  design  organizations.  Continuity  is  one 
of  the  most  important  key  words  in  this  business. 
[10:1327] 

Similar  concern  over  continuity  is  shared  by  the  Naval 
Air  Systems  Command  (NAVAIRSYSCOM) .  In  a  presentation  on 
"NAVAIR  Program  Analysis  and  Evaluation  (Past  Experience) " 
to  the  Naval  Aviation  Executive  Institute,  Mr.  George  A. 
Spangenburg  expressed  his  concern  for  the  loss  of  technical 
continuity  within  the  Systems  and  Engineering  Group  (NAVAIR- 
05)  of  the  NAVAIRSYSCOM.  [25]  Spangenburg,  who  retired 


.  - 
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from  NAVAIR-05  as  a  technical  director  in  the  mid-1970s, 
indicated  the  key  factors  resulting  in  continuing  loss  of 
NAVAIR  technical  continuity  include  fewer  new  starts,  long¬ 
term  reduction  in  NAVAIR  manning  levels  for  engineering/ 
technical  specialists,  and  changing  aircraft  weapon  system 
procurement  policies . 

D.  SCARCITY  OF  EXPERIENCED  AEROSPACE  ENGINEERS 

Aerospace  designers  are  becoming  an  "endangered  species" 
in  the  nation's  aerospace  industry.  The  reduction  in  the 
supply  of  experienced  designers  has  been  largely  a  result 
of  fewer  needs  by  industry  and  better  career  opportunities 
in  analytical  specialization. 

Also  contributing  to  the  scarcity  of  engineering 
personnel  was  the  severe  economic  recession  experienced  by 
the  industry  during  the  early  1970s,  and  the  anti-technology 
movement  among  American  youth  inspired  by  the  Vietnam  war, 
both  of  which  were  responsible  for  directing  many  college 
students  away  from  aerospace. 

Termination  of  the  supersonic  transport  development 
program  in  conjunction  with  waning  design  engineering 
requirements  for  the  B747,  DC-10  and  L-1011  wide-body 
programs  produced  large  scale  lay-offs  among  engineers. 

The  first  to  receive  employment  termination  notices  were 
those  personnel  with  least  seniority  and  experience;  thus, 
most  of  the  reduction  in  engineering  manpower  took  place 
among  young  engineers.  Fewer  young  engineers  are  now 
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interested  in  entering  this  industry  typified  by  dramatic 
economic  ups  and  downs.  Consequences  of  this  situation  will 
be  seen  in  the  1980s  when  comparatively  few  experienced 
engineers  will  be  available  to  fill  the  void  created  by 
attrition  of  the  senior  engineering  force.  [17:37] 

The  Boeing  Company  is  already  feeling  the  "sting"  in 
the  shortage  of  aerospace  engineers  required  for  development 
of  its  new  B757  and  B767  jet  airliners.  To  overcome  this 
shortage,  Boeing  has  undertaken  a  unique  method  to  build  up 
its  engineering  staff.  It  is  borrowing  engineers  from  other 
aerospace  contractors  attempting  to  win  lucrative  subcontracts 
in  the  B757  and  B767  programs.  "We  simply  told  them  that 
without  the  engineers  there  would  be  no  subcontract  awards , " 
says  Boeing's  Director  of  Personnel,  Frank  Gregory.  "The 
engineer  shortage  is  so  severe  that  it  was  a  do-or-die 
situation."  [20:49] 

While  Boeing's  great  "engineer  grab"  may  provide  the 
numbers  of  engineers  required  for  the  B757  and  B767  projects, 
it  must  be  recognized  that  many  will  lack  the  "home-grown" 
experience  and  expertise  unique  to  the  design  of  jet  airliners 
upon  which  Boeing  has  built  a  world-wide  reputation. 

E.  LOWER  LEVELS  OF  EXPERIENCE  BUILD-UP 

Lower  comparable  levels  of  experience  build-up  per  unit 
of  time  in  hardware  design  by  junior  design  engineers,  also 
a  result  of  fewer  new  starts,  is  yet  another  concern. 
Attainment  of  the  many  practical  considerations  and  skills 
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for  application  in  the  hardware  design  phase  of  the  design 
process  comes  primarily  from  experience.  Dr.  Rene  H.  Miller, 
MIT's  Aeronautics  and  Astronautics  Department  head,  is  of 
the  opinion  that  the  engineering  involved  in  detailed  hard¬ 
ware  design  requires  less  of  what  is  taught  in  the  universi¬ 
ties  than  of  what  is  taught  by  experience  [12:29].  Without 
repetitive  on-the-job  design  assignments  an  appreciation  of 
past  design  experience  and  its  influence  on  future  design 
efforts  is  most  difficult  to  achieve. 

F.  GREATER  FINANCIAL  RISK 

Historically,  aeronautical  weapon  systems  have  become 
increasingly  more  complex  with  each  passing  generation  of 
aircraft.  For  the  most  part,  this  complexity  is  a  direct 
result  of  incorporating  high  technology  and  more  demanding 
systems  performance  requirements. 

Directly  proportional  and  related  to  the  trend  of 
increasing  complexity  is  the  trend  of  increasing  aircraft 
cost.  It  can  be  shown  that  the  unit  cost  of  military  air¬ 
craft  is  increasing  at  an  exponential  rate.  The  historical 
trend  of  escalating  unit  cost  for  tactical  aircraft  is  shown 
in  Figure  4.  From  the  Wright  Brothers  to  today's  high  tech¬ 
nology,  high  performance  F-18,  the  cost  of  an  individual 
military  aircraft  has  invariable  grown  by  a  factor  of  four 
every  10  years.  [1:63] 

The  financial  risk  associated  with  military  aircraft 
development  programs  lies  more  with  DoD  than  with  the  major 
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manufacturers.  Survival  of  a  development  program  is 
predicated  all  too  often  on  the  outcome  of  the  many  encounters 
with  the  various  steps  in  the  budget  cycle. 


The  commercial  air  transport  industry  has  seen  similar, 
if  not  greater  cost  escalations  in  its  flight  equipment. 
Despite  lesser  levels  of  technology  required  of  transport 
aircraft,  due  to  the  absence  of  advanced  state-of-the-art 
weapons  fire-control  and  delivery  systems  required  on 
military  aircraft,  unit  cost  for  today's  typical  wide-body 
is  in  excess  of  35  million  dollars  as  compared  to  unit  cost 
of  several  hundred  thousand  dollars  for  the  typical  transport 
of  the  1930s.  Development  costs  for  the  current  generation 
of  American-built  wide-body  transports  has  been  a  staggering 
$1.0  to  $1.5  billion  for  each  design  produced  (B747,  DC-10, 
L-1011) .  Such  figures  and  the  cash-flow  typical  of  commercial 
development  programs  as  shown  in  Figure  5,  clearly  portrays 
the  nature  of  financial  risk  undertaken  by  the  prime  contrac¬ 
tors,  subcontractors,  suppliers  and  the  supporting  banking 
community. 

The  risk  for  the  major  manufacturers  is  centered  on  the 
probability  of  recovering  massive  nonrecurring  costs  for  air¬ 
frame  and  propulsion  system  design,  development,  and  Federal 
Aviation  Administration  (FAA)  certification.  Longer  produc¬ 
tion  runs  (larger  sales  volume)  are  now  required  to  recover 
costs  and  make  a  profit. 
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G.  INCREASING  ENGINEERING  CHANGE  PROPOSALS  EXPENDITURES 

Increasing  aircraft  complexity  has  precipitated  yet 

another  trend,  that  of  increasing  expenditures  for  modifica¬ 
tions  to  production  military  aircraft.  These  modifications 
are  a  direct  result  of  Engineering  Change  Proposal  (ECP) 
action  by  both  the  military  user  and  the  manufacturer. 

Lacking  statistical  information  concerning  the  number  of  ECP 
actions,  the  Naval  Weapons  Engineering  Support  Activity 
(NWESA)  directed  study  efforts  at  financial  expenditures  for 
modifications  resulting  from  incorporating  ECP  in  naval 
aircraft.  If  the  deviant  behavior  associated  with  the 
Vietnam  War  is  ignored,  expenditures  for  ECP  actions  have 
increased  steadily  since  1963.  Through  linear  extrapolation, 
NWESA  projected  ECP  expenditure  to  $0.9  million  and  $1.4 
million  respectively  per  aircraft  for  1985  and  1995  in  1977 
dollars,  an  increase  of  75%  and  153%  respectively  over  1977 
expenditures.  [18:37] 

H.  ERA  OF  COMPUTER-AIDED  DESIGN 

Rapid  advancements  in  computer  technology  of  the  1970s 
has  made  possible  rapid  advancement  of  analytical  and  graphical 
tools  for  computer-aided  design  (CAD) .  Through  interactive 
software  packages,  it  is  no  longer  necessary  to  build  hard¬ 
ware  prototypes  to  investigate  performance,  structures, 
reliability,  maintainability  and  cost.  The  designer  is  now 
able  to  develop  computer  models  of  hardware  solutions  and 
obtain  visual  animated  answers  to  design  questions. 
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CAD  has  become  an  extremely  complex  process  involving 
interrelationships  among  design  phases,  design  applications 
and  design  functions.  Just  recently,  interactive  programs 
for  applying  operational  experience  have  been  made  available 
to  assist  in  the  design  function. 

I.  EMPHASIS  ON  SYSTEM  OPTIMIZATION 

History  shows  that  systems  performance  has  been  the 
traditional  driving  force  behind  military  aircraft  develop¬ 
ment  programs.  This  push  for  performance,  almost  to  the 
exclusion  of  all  other  system  characteristics,  has  resulted 
in  tactical  aircraft  which  have  poor  reliability  and  are 
difficult  and  costly  to  maintain.  These  two  factors,  more 
than  any  others,  have  resulted  in  unacceptable  levels  of 
operational  readiness  and  aircraft  availability  in  fleet 
operating  units. 

The  air  transport  industry  had  long  recognized  the  need 
for  an  optimal  balance  between  aircraft  performance,  relia¬ 
bility,  maintainability  and  life-cycle  costs  if  they  were  to 
remain  a  viable  industry.  By  the  late  1960s,  the  military 
services,  in  view  of  rapidly  growing  operating  and  support 
costs  coupled  with  poor  operational  readiness,  began 
addressing  design  goals  for  systems  optimization.  Since  the 
early  1970s,  major  design  emphasis  has  gone  into  reliability 
and  maintainability  (R&M)  for  such  optimization.  R&M  have 
been  acknowledged  as  primary  life  cycle  cost  "drivers"  which 
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directly  affect  the  operation  and  support  costs  for  aircraft, 
military  and  commercial,  throughout  their  service  lives. 

J.  CHANGING  DoD  SYSTEMS  ACQUISITION  POLICY 

The  military  aerospace  industry  has  been  experiencing 
yet  another  change  during  the  past  several  years.  In  April 
1976,  the  Office  of  Management  and  Budget  COMB)  issued  new 
policies  for  the  acquisition  of  major  systems  by  all  execu¬ 
tive  branch  agencies.  Essentially,  the  new  policies,  OMB 
Circular  A-109,  strive  to  assure  effectiveness  and  effi¬ 
ciency  in  the  process  of  acquiring  major  systems.  [19:3-4] 
Policies  thought  to  have  significant  DoD/Industry  interface 
in  acquisition  of  future  aircraft  weapon  systems  are  those 
that  direct  the  acquiring  agency  to: 

1.  Express  needs  and  program  objectives  in  mission 
terms  rather  than  specific  capabilities  or 
equipment  requirements  to  encourage  innovation 
and  competition  in  operating,  exploring  and 
developing  alternative  system  design  concepts; 

2.  Place  emphasis  on  the  initial  activities  of  the 
system  acquisition  process  to  allow  competitive 
exploration  of  alternative  system  design  concepts 
in  response  to  mission  needs;  and 

3.  Rely  on  private  industry  for  its  needs  for  products 
and  services  in  accordance  with  the  policy  estab¬ 
lished  by  OMB  circular  A-76. 
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While  A-109  may  well  improve  the  efficiencies  associated 
with  aircraft  acquisition  schedule  and  cost,  the  benefits  to 


be  derived  by  reducing  the  customer's  and  ultimate  user's 
role  in  system  and  subsystem  design  may  detract  from  operator 
service  suitability.  In  the  past,  time-consuming  reconcilia¬ 
tion  of  opinions  based  on  experience  by  operators, 
NAVAIRSYSCOM  staffs,  in-house  Navy  laboratories  and  industry 
had  a  positive  effect  in  the  design  decision  making  process. 

Under  A-109,  utilization  of  operational  experience  is 
questionable.  The  concept  and  approach  to  design  will  now 
be  developed  competitively,  using  relatively  short-term 
contracts  in  response  to  solicitations  which  express 
end-objective  functions  a  bidding  contractor's  system  must 
perform.  Contractors  are  forced  by  competition  to  follow 
disciplined  processes  to  deliver  and  justify  their  best 
design  proposals  on  a  timely  basis.  [5:13] 
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III.  ROLE  OF  OPERATIONAL  EXPERIENCE 


The  basic  trends  encountered  by  aerospace  design  organi¬ 
zations  during  the  1970s  have  been  discussed  in  the  previous 
section  of  this  study.  Many  of  these  trends  are  thought  to 
have  a  negative  or  deteriorating  effect  on  aerospace  design 
capability.  These  trends,  when  coupled  with  a  less  than 
optimum  flow  of  operational  experience  from  the  user  or 
operator  back  to  the  front  end  of  new  hardware  design,  further 
detract  from  design  capability. 

This  section  first  investigates  practical  know-how  as 
the  ingredient  essential  to  good  hardware  design.  It  then 
relates  the  importance  of  experience  feedback  as  a  means  to 
preserve  aircraft  design  expertise  in  government  and  industry 
new  development  programs. 

A.  THE  DESIGN  PROCESS 

The  design  process  consists  of  a  hierarchical  sequence, 
beginning  with  ideas  and  concepts  and  ending  with  manufacturing 
instructions  and,  if  necessary,  government  product  certifica¬ 
tion.  Within  this  sequence,  hardware  design  correlates  best 
with  the  practical  aspects  of  operational  experience  and  is 
the  event  that  deserves  most  attention  by  this  study.  Haupt, 
in  discussing  design  hierarchy,  describes  hardware  design  as 
follows:  [17:8-9] 
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Within  the  constraints  imposed  by  the  chosen 
configuration,  it  is  concerned  with  the  actual 
hardware  of  structures  and  systems.  Alternative 
solutions  and  tradeoffs  have  to  be  considered  just 
as  in  parametric  studies  —  but  now  the  practical 
level  of  hardware  replaces  the  abstract  level  of 
parameters.  Each  component  part  is  analyzed 
from  the  separate  viewpoints  of  function,  strength, 
cost,  reliability,  producibility ,  maintainability, 
etc.  Structural  and  functional  tests  may  be 
required  for  verification.  The  emphasis  is  on 
engineering  know-how. 

Hardware  design  can  also  be  thought  of  as  a  conversion 
process  —  converting  quantitative  input  requirements  into 
qualitative  design  solution  outputs.  Application  of  practical 
know-how  is  the  primary  element  in  the  conversion  process. 
Illustrative  of  the  hardware  design  event  in  the  design 
process  is  the  example  shown  in  Figure  6.  It  portrays  the 
typical  process  in  gross  terms  as  applied  to  landing  gear 
design.  The  success  or  failure  of  hardware  design  in  terms 
of  systems  performance  and  life  cycle  costs  is  keyed  to 
designer  know-how. 

Know-how,  ideally  the  blending  of  latest  technology, 
existing  engineering  knowledge  and  operational  experience, 
is  what  produces  optimum  hardware  configuration.  Of  these 
three  forms  of  engineering  design  know-how,  operational 
experience  can  only  be  utilized  through  very  close  liaison 
between  designer  and  user  shown  in  Figure  7.  This  is 
perhaps  the  weakest  link  in  the  chain  of  design  responsibility. 
Theory  indicates  that  this  link  is  essential  for  optimal 
hardware  design.  However,  even  where  good  liaison  seems  to 
exist  between  customer  and  manufacturer,  it  has  been  concluded 
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LANDING  GEAR  DESIGN  EXAMPLE 
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OPERATIONAL  EXPERIENCE  FEEDBACK --LINK  BETWEEN  USER  AND  HARDWARE 

DESIGNER. 


that  operational  experience  is  not  necessarily  adopted 
in  new  design.  [4:10] 

B.  VALUE  OF  OPERATIONAL  EXPERIENCE 

Build-up  of  design  know-how  in  the  area  of  operational 
experience  not  only  assists  contractors  in  achieving  a 
competitive  edge  in  the  marketplace,  but  more  importantly 
benefits  the  customer  by  potentially  maximizing  measures  of 
systems  effectiveness  (i.e.  operational  readiness,  aircraft 
availability,  reliability,  maintainability)  and  minimizing 
total  life  cycle  costs.  When  compared  to  Naval  Air  Systems 
Command  (NAVAIRSYSCOM)  aircraft  development  programs, 
commercial  aircraft  development  endeavors  have  historically 
pursued  more  vigorously  the  utilization  of  operational 
experience  because  of  more  cost-conscious  airline  customers. 

In  the  past,  systems  performance  has  been  the  driving 
force  behind  military  aircraft  development  with  little  or 
no  regard  for  life  cycle  costs  and  systems  effectiveness. 

This  emphasis  on  performance  often  displaced  proven  state- 
of-the-art  technology  and  operational  experience  for  unproven 
advanced  technology.  There  was  little  need  or  incentive  to 
look  back  and  learn  from  operational  experiences.  Penalties 
have  been  excessive  operating  and  support  costs,  and 
unacceptable  levels  of  operational  readiness  and  aircraft 
availability  in  fleet  squadrons. 

It  has  not  been  until  recent  times  that  military  air¬ 
craft  design  priorities  have  been  re-directed  to  specifying 
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required  levels  of  achievement  for  systems  effectiveness. 

Most  new  development  contracts  now  include  incentive/penalty 
clauses  for  systems  effectiveness  parameters.  At  minimum  the 
parameters  express  required  levels  of  operational  readiness, 
aircraft  availability,  and  major  sub-system  reliability  and 
maintainability . 

Systems  effectiveness  parameters  to  be  achieved  by  the 
aircraft  weapons  system  are  specified  in  quantitative  terms. 
It  is  then  the  designer's  responsibility  to  convert  the 
specified  quantitative  requirements  into  qualitative  hardware 
design  characteristics  capable  of  meeting  or  exceeding  the 
specified  requirements.  To  accomplish  this  task,  designers 
must  draw  very  largely  on  past  design  knowledge. 

C.  THE  EXPERIENCE  GAP 

The  preferred  method  by  which  previous  design  knowledge 
can  be  obtained  is  by  first-hand  experience  from  previous 
hardware  design  assignments.  Frequent  new  development  starts 
from  post  WWII  until  the  mid  1950s  provided  much  experience 
for  aeronautical  designers  of  that  time.  The  experience 
build-up  of  that  decade  undoubtedly  assisted  many  design 
teams  in  creating  better  alternative  solutions  to  design 
problems  in  subsequent  design  efforts.  Hopefully  such 
experience  also  avoided  repetitive  mistakes  of  previous 
designs . 

The  most  recent  generation  of  aerospace  design  engineers 
have  not  had  similar  opportunities  for  obtaining  such 


experience  build-up  and  resultant  knowledge.  The  average 
level  of  design  experience  build-up  achieved  by  the  individual 
engineer  is  steadily  declining,  a  result  of  fewer  program 
starts  and  attrition  of  senior  design  engineers.  Yet, 
emphasis  since  the  mid-1960s  for  improving  life  cycle  costs 
and  system  effectiveness  compounded  by  increased  systems 
complexity  has  given  cause  to  achieve  a  higher  level  of 
experience  build-up.  Divergence  between  acquired  and  needed 
levels  of  design  experience  build-up  has  been  expanding  the 
experience  gap,  shown  in  Figure  8. 

D.  FEEDBACK  FOR  SHRINKING  THE  EXPERIENCE  GAP 

As  early  as  1971,  Dr.  John  S.  Forster,  Jr.,  then  Director 
of  Defense  Research  and  Engineering,  strongly  expressed  his 
viewpoints  to  recreating  and  preserving  design  expertise  in 
government  and  industry.  [21:21]  Little  has  been  accomplished 
by  the  DoD  over  the  ensuing  years,  in  way  of  a  unified 
approach  to  restrain  the  growth  in  the  experience  gap  and 
preserve  design  expertise  in  military  aircraft  development 
programs.  Nevertheless,  industry  long  recognizing  the  need 
to  find  ways  of  obtaining  operational  experience,  has  been 
taking  progressive  steps  in  developing  suitable  experience 
feedback  systems.  [23:103] 

In  light  of  today's  rapidly  expanding  computer  informa¬ 
tion  processing  capabilities,  computer-aided  engineering 
design  feedback  systems  can  be  made  a  reality.  Such  systems 
could  greatly  assist  in  shrinking  the  experience  gap. 
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••  GROWING  EXPERIENCE  GAP  IN  AEROSPACE  DESIGN. 


However,  to  achieve  maximum  benefit,  the  systems  must  be 
formulated  and  developed  to  provide  operational  experience 
information  in  a  timely  and  meaningful  user  language  format. 
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IV.  FEEDBACK  OF  OPERATIONAL  EXPERIENCE 
IN  COMMERCIAL  AIR  TRANSPORTATION 

Apart  from  the  past  major  technological  breakthroughs 
(turbojet  engine,  swept  wing,  high-by-pass  fan  engine,  etc.) 
the  overall  gain  by  the  air  transport  industry  has  been 
achieved  by  a  large  number  of  relatively  small  improvements 
in  numerous  areas.  Hence,  technology  advances  have  not 
necessarily  been  stored  and  used  all  at  once  in  a  new  design, 
with  all  the  attendant  risks,  but  rather  they  have  been 
incorporated  gradually  in  successive  aircraft  development 
programs.  [3:462]  Many  of  the  small  improvements  result 
from  the  lessons  learned  in  daily  operations.  Aerospace 
contractors  associated  with  the  air  transport  industry  find 
feedback  of  operational  experience  from  airline  customers 
essential  for  continued  improvements  in  present  production 
and  new  development  aircraft.  Figure  9  illustrates  the 
several  important  information  sources  providing  feedback 
from  the  customer  (ultimate  user)  to  the  manufacturer  in  the 
commercial  air  transport  industry.  It  should  be  noted  that 
contractor  technical  information  systems  and  field  service 
representatives  are  internal  or  first  level  feedback  sources, 
while  Federal  Aviation  Administration  (FAA)  and  the  Air 
Transport  Association  (ATA)  are  external  or  second  level 
sources . 
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AIRCRAFT  INFORMATION  FEEDBACK  CUSTOMER/ 

CONTRACTOR  MEDIA  USER 


Figure  9  OPERATIONAL  EXPERIENCE  FEEOBACK  FLOW -- IN  COMMERCIAL  AIR 
TRANSPORTATION. 


A.  INTERNAL  OR  FIRST  LEVEL  SOURCES 

1.  Management  Information  Systems 

One  method  by  which  aircraft  contractors  can  maintain 
liaison  with  the  airline  operators  is  via  some  form  of  a 
Management  Information  System  (MIS) .  This  is  a  centralized 
system  for  providing  information  to  support  current  opera¬ 
tions,  future  aircraft  development,  program  management, 
and  organizational  decision-making  functions. 

Each  of  the  three  principal  domestic  transport  air¬ 
craft  manufacturers  has  its  own  unique  system.  Lockheed- 
California  Company  (LCC)  will  be  taken  as  an  example.  Their 
management  information  system,  called  Operational  Support 
Data  System  (OSDS) ,  has  been  providing  feedback  information 
since  1969. 

OSDS  is  essentially  a  closed  loop  information  feed¬ 
back  system.  At  the  time  of  its  inception,  its  primary 
purpose  was  to  provide  visibility  to  program  management  that 
the  L-1011  TriStar  is  performing  to  the  required  levels  of 
reliability,  maintainability,  safety  and  performance  standard. 
Responsibility  for  operation  of  the  OSDS  was  assigned  to 
Lockheed's  Commercial  Product  Support  Center.  Functions 
of  this  center  are  listed  in  Figure  10,  and  include  design 
support.  Commercial  reliability  and  maintainability 
responsibilities  of  the  support  center  are  spelled  out  in 
Lockheed's  Management  Directive  Number  162  contained  in 
Appendix  B  of  this  study. 
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A  lesser  priority  purpose  of  the  now  ten  year  old 
OSDS  was  the  feedback  and  build-up  of  design  experience 
information  related  to  the  L-1011  TriStar.  The  current 
value  of  this  information  is  now  priceless  in  Lockheed's 
on-going  TriStar  product  improvement  programs,  L-1011 
derivative  development  programs,  and  preliminary  design 
efforts  for  a  TriStar  replacement. 

LCC  employs  OSDS  in  two  ways  —  in  everyday  opera¬ 
tions  and  in  technical  management  decision-making.  Design- 
oriented  functions  are  an  important  part  of  everyday 
operations.  As  defined  by  Ein-Dor  and  Segev: 

A  management  information  system  is  a  system  for 
collecting,  storing,  retrieving  and  processing 
information  that  is  used,  or  desired,  by  one  or 
more  managers  in  the  performance  of  their 
duties.  [6:16] 

In  view  of  this  generally  accepted  definition  of  MIS,  LCC’s 
OSDS  can  very  well  be  titled,  "A  Technical  Management 
Information  System  (TMIS)". 

a.  Information  Inputs 

In  the  customer  and  contractor  interface,  feedback 
emerges  from  one  source  only  --  the  customer,  as  shown  in 
Figure  9.  When  considering  the  user  and  designer  interface 
within  the  respective  customer  and  contractor  organizations, 
feedback  emerges  from  a  multitude  of  sources.  Sources 
include  the  contractor's  test  and  evaluation  activities,  and 
his  subcontractors  and  suppliers  for  furnished  equipment  and 
hardware.  Four  sources  of  feedback  in  LCC's  OSDS  are  shown 
in  Figure  11,  under  data  sources. 
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OATA  SOURCES  DATA  CONTROL  DATA  APPLICATIONS 


FIGURE  II:  LOCKHEED'S  OPERATIONAL  SUPPORT  DATA  SYSTEM  (OSDS) 


r 


Source  data  can  be  made  available  in  a  variety 
of  forms,  depending  upon  circumstances  and  agreements  pre¬ 
viously  reached  with  the  sources.  Due  to  differences  in 
reporting  capabilities  and  differences  encountered  in  world 
wide  airline  operations,  vehicles  for  reporting  must  remain 
flexible.  Lockheed,  for  example,  processes  every  kind  of 
input  from  text  written  on  paper  through  automated  data 
processing  punch  cards  and  magnetic  tapes . 

Standardization  of  source  data  inputs  is  most 
important  in  standardization  of  the  data  bank  baseline. 
Quantitative  data  inputs  such  as  aircraft  utilization, 
departure  delays,  maintenance  cost,  and  the  like,  require 
standardized  measurement  by  all  participating  data  sources. 
Failure  to  do  so  will  result  in  an  unreliable  and  non-usable 
output  product.  Lockheed  reports  that  achieving  uniformity 
in  reporting  by  all  data  feedback  sources  has  been  and 
continues  to  be  the  greatest  difficulty  in  their  OSDS, 
b.  Information  Processing 

The  major  portion  of  a  management  information 
system  is  concerned  with  collecting,  reducing,  storing, 
retrieving,  analyzing  and  disseminating  information.  The 
aggregate  of  these  activities  can  be  termed  information 
processing.  Lockheed  chooses  to  call  it  data  control. 

Data  control  within  Lockheed's  OSDS  is  also  the 
responsibility  of  the  Commercial  Product  Support  Center. 

As  shown  in  Figure  11,  the  elements  of  data  control  conform 
to  the  intent  of  information  processing  described  for  a  MIS. 
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c.  Output  Products 

The  type  and  number  of  MIS  output  products  are 
completely  variable  and  limited  only  by  source  data  and 
processing  software  programs  capabilities.  However,  if  the 
MIS  is  to  be  successful,  the  products  must  be  compatible 
with  the  needs  and  requirements  of  the  users. 

The  output  from  Lockheed's  OSDS  comes  in  various 
forms  and  covers  a  variety  of  subjects  in  reports  tailored 
to  meet  the  requirements  of  L-1011  TriStar  management  tasks. 
The  output  reports  listed  in  Figure  11  are  typical  of  the 
periodic  reports  processed  by  OSDS.  A  representative  sample 
of  output  reports  related  to  operational  experience  is 
contained  in  Appendix  C. 

d.  Information  Applications 

Among  the  criteria  for  determining  a  successful 
from  an  unsuccessful  MIS  is  how  the  output  products  are 
applied.  One  condition  for  a  successful  MIS  is  that  it  be 
applied  to  major  problems  of  an  organization.  [6  : 19—20] 

Broadly  speaking,  OSDS  programs  provide  the 
necessary  information  for  three  areas  of  major  concern: 
First,  keeping  upper  and  operating  management  informed  of 
program  performance  status;  second,  establishing  priorities 
for  directing  product  improvement;  and  third,  providing 
technical  groups  (product  support  and  engineering)  design 
experience  with  fielded  hardware.  It  is  the  third  area  that 
is  of  concern  in  this  study. 
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Lockheed's  OSDS  finds  definite  application  in 
monitoring  design  deficiencies  or  problem  areas  as  they  show 
up  in  airline  operation.  When  a  transport  aircraft  is  unable 
to  meet  a  scheduled  departure  for  technical  reasons,  it  is 
essential  to  have  immediate  feedback  of  engineering  informa¬ 
tion  and  to  follow  up  with  remedial  action.  The  corresponding 
organization  is  provided  by  Product  Support.  It  is  based 
on  availability  of  operational  experience.  [22:11] 

A  most  important  aircraft  performance  parameter 
for  airline  operations  is  dispatch  reliability.  Dispatch 
delays  in  excess  of  15  minutes  or  flight  cancellation  are 
closely  monitored  as  to  cause.  Aircraft  systems  contributing 
to  dispatch  delays  or  cancellations  are  monitored  and  reported 
by  OSDS.  Standard  reports,  as  shown  in  Figure  12,  rank 
system  contribution  by  percent  of  total.  Lockheed's  effort 
to  improve  L-1011  dispatch  reliability  serves  as  an  example 
of  OSDS  design  support  application  in  both  special  studies 
and  tackling  in-service  technical  problems. 

Using  OSDS  data  bank  information,  subsequent 
special  studies  are  undertaken  to  define  the  problem  and 
identify  the  problem  component.  After  the  component  has 
been  identified,  in-depth  and  detailed  studies  follow  to 
determine  component  failure  item  and  cause  of  failure. 

These  data  and  information  are  then  transferred  from  the 
OSDS  operations  group  to  program  engineering.  Engineering 
determines  the  "why"  of  failure  and  develops  a  new  or 
improved  design  for  inclusion  into  existing  and  future 
programs . 
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FIGURE  12:  AIRCRAFT  SYSTEM  CONTRIBUTION  TO  DISPATCH  DELAYS  AND  CANCELLATIONS 


2.  Field  Service  Representatives 


Since  the  days  of  the  Wright  Flyer,  a  vital  link 
between  the  contractor  and  his  customers  has  been  the  Field 
Service  Representative,  often  better  known  as  the  "Tech-Rep." 

In  the  early  days,  Tech-Reps  were  just  what  the  name  implied  — 
Technical  Representatives.  However,  in  today's  environment, 
they  provide  a  host  of  administrative  as  well  as  technical 
services.  Services  are  rendered  to  both  their  employer  and 
customer. 

The  vast  majority  of  data  inputs  for  technical 
management  information  systems,  such  as  Lockheed's  Operational 
Support  Data  System,  are  generated  by  Field  Service  Representa¬ 
tives.  L-1011  TriStar  field  reports  generated  for  inclusion 
in  the  OSDS  are  found  in  Table  I.  Volume  of  communications 
traffic  associated  with  these  reports  from  19  field  offices 
(11  American  and  8  International)  is  shown  in  Figure  13. 

Field  Service  Representatives  are  also  the  contractor's 
eyes  and  ears.  Aside  from  the  mundane  tasks  of  providing 
technical  and  administrative  assistance  to  both  their  employer 
and  customer.  Field  Service  Representatives  are  in  a  position 
to  compile  competitive  business  intelligence. 

The  intelligence  function  in  business  is  recognized 
by  many  competitive  corporations  as  one  of  the  keys  for 
successful  strategic  planning  and  decision  making  by  top 
management.  [26:2]  Intelligence  gathering  by  Field  Service 
Representatives  accounts  for  only  a  small  segment  of  business 
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intelligence  assemination,  and  should  be  considered  confined 
to  areas  of  technical  interest. 

Areas  of  technical  interest  include  monitoring  the 
performance  characteristics  of  competitor  equipment,  observing 
airline  operator  likes  and  dislikes  in  both  equipment  being 
represented  and  competitor  equipment,  and  identifying 
desirable  features  in  operator's  future  equipment  needs. 
Feedback  and  utilization  of  such  intelligence  can  be  of  great 
assistance  in  directing  design  strategy  to  be  taken  in  new 
aircraft  developments,  and  in  tailoring  future  equipment  to 
the  airline  operator  needs  and  environment. 

B.  EXTERNAL  OR  SECOND  LEVEL  SOURCES 

1 .  Air  Transport  Association  of  America  (ATA) 

The  ATA  is  a  membership  organization  serving  the  many 
needs  of  U.S.  flag  air  carriers.  While  ATA's  emphasis  in 
support  of  air  carrier  needs  is  on  federal  regulatory  affairs, 
it  also  provides  engineering  and  maintenance  services.  Of 
interest  are  its  engineering  and  maintenance  activities  keyed 
to  aircraft  design  standardization  in  areas  of  ground 
servicing  and  maintenance,  technical  information  index 
standardization,  technical  problems  conferences,  and  developing 
the  industry's  position  on  Federal  Aviation  Administration 
proposed  rule  making.  The  unified  position  on  technical 
issues  voiced  by  the  ATA  for  the  air  transport  industry  has 
definite  influential  impact  on  federal  air  regulation  rule 
making  and  contractor  response  to  in-service  equipment 
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problems  as  well  as  future  derivative  or  next  generation 
aircraft  design  characteristics  and  features. 

The  feedback  of  design  or  technical  information  from 
the  airline  operators  to  the  contractor's  design  engineering 
team  via  the  ATA  is  shown  in  Figure  9  and  is  external  to 
normal  channels  employed  by  the  contractor.  Information 
feedback  of  this  type  is  primarily  generated  by  "as  the  need 
arises"  or  "on-exception"  basis. 

2 .  Federal  Aviation  Administration  (FAA) 

The  Federal  Aviation  Administration  (FAA) ,  an  agency 
of  the  Department  of  Transportation,  has  comprehensive 
authority  over  air  safety  and  control  over  all  U.S.  airspace. 
FAA's  technical  responsibilities  in  the  air  transport  industry 
include:  establishing  and  maintaining  Federal  Air  Regulations 

related  to  requirements  for  aircraft  design  and  construction; 
aircraft  airworthiness  certification;  monitoring  health  of 
aircraft  fleets;  approving  major  engineering  changes  and 
alterations  to  civil  aircraft;  and  issuing  airworthiness 
directives  for  mandating  action  required  by  contractors  and 
operators  for  sustaining  aircraft  airworthiness.  FAA  air- 
carrier  field  office  activities  at  airline  main-base  of 
operations  and  mandatory  airline  reporting  of  equipment 
defects  or  failures  adversely  affecting  flight  safety  are 
the  primary  sources  of  information  feedback. 

The  flow  of  information,  like  that  of  the  ATA,  is 
external  to  the  regular  or  normal  channels  employed  by  the 
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contractor  and  it  too  is  predicated  on  "as  the  need  arises” 
or  an  "on-exception"  basis. 
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V.  FEEDBACK  OF  OPERATIONAL  EXPERIENCE 
IN  NAVAL  AVIATION 

Excluding  military  avionics  systems,  technology  advances 
in  applied  military  aerospace  technology  have  historically 
been  the  "pathfinders"  for  the  entire  nerospace  industry. 

In  the  past,  the  push  for  superior  preformance  requirements 
in  military  programs  contrasted  sharply  with  life-cycle 
costs  and  systems  effectiveness  objectives  of  the  air 
transport  industry.  Changes  in  Naval  Air  Systems  Command 
(NAVAIRSYSCOM)  acquisition  policy  have  placed  increased 
emphasis  on  life-cycle  costs  and  systems  effectiveness. 

Despite  efforts  to  improve  weapon  system  acquisition 
efficiency  and  effectiveness  through  a  pyramid  of  ever- 
changing  program  management  policies  and  directives,  the 
single  most  dominant  factor  governing  life-cycle  costs  and 
systems  effectiveness  relates  to  operational  experience. 

A  clearly  established  system  for  feeding  back  operational 
experience  to  the  designer  can  greatly  assist  aircraft 
contractors  in  achieving  NAVAIRSYSCOM  aircraft  design 
objectives . 

A.  CONTRACTOR  INFORMATION  FEEDBACK  SOURCES 

Figure  14  portrays  the  primary  sources  of  feedback  from 
the  Navy  user  to  the  prime  contractor.  Although  information 
feedback  sources  closely  resemble  sources  in  commercial 
programs  shown  in  Figure  9,  vast  differences  exist  in 


information  flow.  The  differences  are  largely  a  result  of 
decentralized  information  collection,  storage,  retrieval, 
analysis  and  application. 

In  military  aircraft  programs,  little  emphasis  has  been 
placed  on  systematic  evaluation  of  experience  until  recently. 
There  are  several  reasons  for  this.  First,  with  roughly 
10  major  aircraft  manufacturers  (fixed  and  rotor  wing) 
competing  for  less  than  half  as  many  development  contracts 
in  a  given  10  year  span,  there  is  only  remote  probability 
in  favor  of  landing  a  development  contract  as  prime 
contractor.  Under  such  odds,  little  incentive  exists  to 
develop  and  maintain  a  feedback  capability  and  data  bank, 
especially  when  sustaining  engineering  funds  are  not  pro¬ 
vided  by  NAVAIRSYSCOM.  Contrary  to  the  airline  operator’s 
farsighted  procurement  policy,  military  services  depend  on 
yearly  approvals  of  a  Congressional  budget.  This  has  had 
an  adverse  effect  on  the  long-range  funding  required  for  a 
feedback  system.  There  are,  however,  signs  for  improvements 
in  the  future.  A  second  reason  stems  from  industry  competi¬ 
tion  differences  between  military  and  commercial  aircraft 
procurement.  Compared  to  military  sole-source  procurements, 
the  airlines  are  at  liberty  to  choose  from  a  variety  of 
similar  and  competing  aircraft.  To  hopefully  gain  a 
competitive  edge  in  commercial  programs,  a  contractor  will 
take  full  advantage  of  operational  experience  from  fielded 
equipment. 
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From  Figure  14,  the  only  internal  or  first  level  feedback 
source  available  to  the  contractor  is  through  his  field 
service  representatives .  Three  of  the  remaining  four  sources 
are  external  or  second  level  and  one  source  is  direct  from 
NAVAIRSYSCOM . 

1 .  Internal  or  First  Level  Sources 

Since  the  fledgling  days  of  military  aviation, 
contractor  field  service  representatives  have  provided  the 
armed  forces  invaluable  technical  services.  Their  technical 
assistance  to  improve  the  technical  expertise  of  military 
maintainers  has  had  a  positive  impact  on  achieving  operational 
readiness  goals  and  is  considered  one  of  their  primary  roles. 
Another  primary  role  pertains  to  monitoring  performance  and 
systems  effectiveness  of  the  aircraft  weapon  system  they 
represent. 

Significant  technical  problems  or  discrepancies 
encountered  are  reported  via  numerous  modes  of  communication. 
Contractor  utilization  of  such  feedback  data  is  usually 
confined  to  resolving  current  product  problems  and  product 
improvement,  whereas  in  commercial  programs,  such  data  would 
also  be  utilized  in  future  derivative  or  new  development 
projects . 

In  the  late  1960s,  the  Naval  Aviation  Engineering 
Services  Unit  (NAESU)  sought  self-sufficiency  for  the  Navy 
during  the  latter  phase  of  the  aircraft  life-cycle  coupled 
with  the  desire  to  reduce  expenditures  for  contractor 
sustaining  engineering.  The  result  was  creation  of  the 
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Contractor  Engineering  Technical  Services  (CETS)  and  Navy 
Engineering  Technical  Services  (NETS)  programs. 

Under  the  CETS/NETS  concept,  contractor  field-service 
representatives  assigned  to  fleet  operating  units  are  fewer 
in  number  and  are  providing  technical  services  for  a  much 
shorter  period  of  time.  What  was  once  a  "cradle-to-grave" 
field-service  support  program  by  the  contractor  has  been 
reduced  to  only  a  fraction  of  the  weapons  system  life  cycle. 

The  effect  on  the  contractors  has  been  a  sizable 
reduction  in  feedback  of  internally  generated  or  first  level 
design  experience  information.  The  end  result  is  undoubtedly 
contributing  to  the  problem  of  maintaining  contractor  design 
team  capability. 

2 .  External  or  Second  Level  Sources 

a.  Navy  Maintenance  Support  Office 

An  excellent  source  of  operational  experience 
data  is  provided  by  the  Navy  Maintenance  Support  Office  (NAMSO) 
of  the  Fleet  Material  Support  Office  (FMSO) .  Designed, 
developed  and  operated  to  provide  maintenance  and  maintenance 
support  data  to  all  levels  of  Navy  management,  NAMSO* s  wealth 
of  analytical  and  statistical  products  are  available  to  all 
authorized  research  and  development  contractors. 

Shown  in  Figure  14  is  NAMSO* s  fit  into  the 
feedback  flow  pattern,  between  the  user  and  contractor.  Its 
input,  processing  and  output  functions  are  similar  to 
Lockheed's  Operational  Support  Data  System  (OSDS)  discussed 
in  Section  IV. 
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(1)  Information  Inputs .  Information  feedback 
for  NAMSO  emerges  from  basically  two  sources  —  fleet  aviation 
activities  and  NAVAIRSYSCOM  research,  development,  test  and 
evaluation  field  activities.  Information  is  accumulated  by 
the  Maintenance  Data  Collection  Subsystem  (MDCS)  which  is 
an  integral  part  of  the  Navy  Maintenance,  Material  and  Man¬ 
power  Management  System,  more  commonly  known  as  the  Navy 
3-M  system.  The  3-M  system  is  sponsored  by  the  Chief  of 
Naval  Operations  (CNO)  and  technically  supported  by  the 
Chief  of  Naval  Material  and  the  Naval  Air  Systems  Command. 

Source  data  is  made  available  only  in  the 
type  and  format  previously  approved  by  CNO  in  OPNAV 
Instruction  4790.2.  Changes  in  data  type  or  format  desired 
by  MSO  must  be  approved  by  CNO.  The  rather  inflexible 
system  does,  however,  have  the  advantage  of  being  a  stabi¬ 
lized  and  standardized  Navy-wide  computerized  data  baseline. 
The  problem  of  data  reduction  to  achieve  a  standardized 
data  base  as  experienced  by  commercial  programs  is  totally 
eliminated  in  the  3-M  system. 

Types  of  quantitative  source  data  reported 
by  the  3-M  system  and  collected  by  MDCS,  applicable  to 
operational  experience,  were  found  to  be  almost  identical 
to  Lockheed's  commercial  data  source  inputs  for  their  OSDS 
listed  in  Figure  11.  Such  3-M  data  include: 

--  MDR  (Maintenance  Data  Reporting) ; 

—  MR  (Material  Reporting) ;  and 

—  ASD  (Aircraft  Statistical  Data)  reporting. 
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(2)  Information  Processing.  Information 


processing  of  3-M  data  by  NAMSO  is  that  typical  of  most 
information  processing  activities  —  storing,  retrieving, 
analyzing  and  disseminating  information. 

(3)  Output  Products.  NAMSO  output  products 
are  numerous  and  users  of  the  numerous  products  are  many. 

The  products  take  the  form  of  standard  reports  and  summaries, 
and  special  requests.  Products  of  interest  to  both  the 
contractors  and  the  NAVAIRSYSCOM  systems  engineering  group 
and  related  to  aircraft  weapon  system  design  experience  are 
contained  in  Appendix  D. 

(4)  Information  Application.  NAMSO  output 
products  serve  a  host  of  applications.  Similar  to  commercial 
program  Technical  Management  Information  Systems  (TMIS) , 

NAMSO  serves  contractor  and  Navy  management  needs  in  three 
broad  areas:  tracking  weapon  system  performance  trends  in 
terms  of  readiness  and  utilization;  establishing  priorities 
for  service  problem  and  deficiency  correction;  and  providing 
technical  groups  information  for  utilization  in  product 
improvement  and  new  design  development  programs.  Unlike 
commercial  program  information  systems,  which  are  designed 
and  operated  solely  for  parent  company  application,  NAMSO 
serves  the  entire  military  aircraft  manufacturing  industry 
and  naval  aviation  management  structure. 

NAMSO  output  products,  quantitative  and 
statistical  in  nature,  find  extensive  application  in  the 
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area  of  weapon  system  effectiveness.  The  recent  Chief  of 
Naval  Material  and  Naval  Air  Systems  Command  push  to  bring 
about  increased  operational  readiness  and  reduced  life  cycle 
cost  in  Naval  aircraft  acquisition  has  given  cause  to  improve 
system  reliability  and  maintainability  (R&M) .  In  response  to 
this  push,  NAVAIRSYSCOM  and  aircraft  contractors  have  looked 
to  NAMSO  for  historical  statistical  data  such  that  R&M 
requirements  and  allocations  can  be  better  predicted  during 
the  early  phases  of  the  acquisition  process, 
b.  Test  and  Evaluation 

Test  and  Evaluation  (T&E)  is  a  vital  function 
in  aircraft  weapon  system  acquisition.  Starting  with  initial 
research  and  development,  T&E  is  an  integral  and  continuing 
part  of  the  acquisition  process.  Essentially,  the  purpose 
of  T&E  is  a  determination  of  the  suitability  of  hardware 
for  service  use.  If  the  hardware  functions  in  a  technically 
acceptable  manner  and  if  the  hardware  meets  the  specified 
operational  performance  requirements  the  hardware  is  con¬ 
sidered  suitable  for  service  use.  [16:27]  Test  and  evalua¬ 
tion  identifies  design  deficiencies  and  enhancing  character¬ 
istics.  Thus,  it  provides  feedback  to  design  teams  and 
program  management  for  the  contractor  and  Navy  alike. 

(1)  The  Process.  Test  and  evaluation  of  a 
typical  current  generation  tactical  aircraft  is  a  long  and 
tedious  process,  often  spanning  six  or  more  years.  The 
process  begins  shortly  after  the  first  flight  with  the  first 
of  several  contractor  demonstrations,  called  Navy  Preliminary 


Evaluations  (NPE) .  The  last  NPE  is  conducted  just  prior 
to  the  Board  of  Inspection  and  Survey  (BIS)  trials. 

The  purpose  of  an  NPE  is  to:  [11:19] 

—  Determine  at  the  earliest  possible 
opportunity  the  combat  potential  and 
gross  deficiencies  of  the  aircraft 
and  its  systems,  and 

—  Allow  early  correction  of  deficiencies. 

Following  the  NPEs,  the  BIS  trials  are 
commenced  to  determine  service  suitability  and  contractor 
specification  conformance  in  production  configured  aircraft. 
All  discrepancies  reported  during  the  NPEs  and  BIS  trials 
must  be  corrected  or  otherwise  resolved  before  the  aircraft 
is  considered  service  suitable. 

Follow-on  T&E  is  conducted  after  the  air¬ 
craft  is  delivered  to  fleet  operating  activities.  The 
majority  of  this  effort  is  confined  to  engineering  change 
proposal  actions,  accelerated  service  testing,  assistance 
in  derivative  development,  and  operating  problems  encountered 
by  fleet  activities. 

(2)  Reporting  Results.  T&E  reporting  provides 
feedback  as  shown  in  Figure  14.  The  Naval  Air  Test  Center 
(NAVAIRTESTCEN) ,  chartered  as  the  Navy's  field  activity  for 
aircraft  weapons  system  T&E,  utilizes  five  basic  report 
types  for  reporting  T&E  results.  The  reports  include: 

—  BIS  Yellow  Sheet  (Y/S) ; 
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—  Deficiency  Report  (DR) ; 

—  Formal  Technical  Report  (TR) ; 

—  Report  of  Test  Results  (RTR) ;  and 

—  Message  Report. 

A  detailed  description  of  these  reports  is  found  in  Table 
II.  With  the  exception  of  the  BIS  Yellow  Sheet  and 
NAVAIRTESTCEN  Deficiency  Report  which  are  confined  to 
deficiency  reporting  only,  aircraft  enhancing  characteristics 
as  well  as  deficiencies  are  reported. 

(a)  Enhancing  Characteristics.  Those 
characteristics  of  the  aircraft  or  equipment  which  signifi¬ 
cantly  enhance  its  operational  use  or  technical  capability 
are  numerated  in  the  report.  The  characteristics  listed 
emphasize  the  significant  strong  points  or  capabilities  and 
provide  the  contractor  and  Navy  program  manager  with  the 
good  side  of  the  picture.  More  importantly  for  design 
experience  information  feedback,  it  acknowledges  outstanding 
characteristics  which  should  be  incorporated  into  future  new 
development  program  designs  and  specifications. 

(b)  Deficiency  Classification.  Deficiencies 
identified  during  the  course  of  T&E  are  fully  qualified  in 
the  text  of  the  applicable  T&E  report.  In  order  to  differ¬ 
entiate  a  more  serious  deficiency  from  a  less  serious  one, 

the  Naval  Air  Test  Center  classifies  each  deficiency. 
Deficiencies  are  placed  into  one  of  the  following  three 


TABLE  II 


TYPES  OF  NAVAL  AIR  TEST  CENTER 
T&E  REPORTS 


1.  BIS  Yellow  Sheet  (Y/S) .  This  is  the  means  for  reporting 
deficiencies  noted  during  Board  of  Inspection  and  Survey 
(BIS)  trials.  The  report  by  its  nature  requires  rapid 
transfer  of  information,  and  as  such,  each  deficiency  is 
reported  separately  on  its  own  report  format. 

2.  Deficiency  Report  (DR) .  This  is  the  means  for  reporting 
deficiencies  noted  during  Navy  preliminary  evaluations 
(NPE) .  The  report  by  its  nature  requires  rapid  transfer 
of  information,  and  as  such,  each  deficiency  is  reported 
separately  on  its  own  report. 

3.  Formal  Technical  Report  (TR) .  This  is  the  primary  means 
for  documenting  the  results  of  project  work.  This  report 
is  used  when  the  reported  results  will  be  significant  for 
a  long  period  of  time,  will  be  widely  distributed  (i.e. 
Defense  Documentation  Center)  or  have  other  important 
aspects  which  require  the  report  to  have  lasting  value. 

4.  Report  of  Test  Results  (RTR) .  This  is  the  means  for 
reporting  information  which  is  limited  in  scope, 
application  or  time.  It  is  usually  used  for  reporting 
interim  results  of  any  project  when  relatively  short 
periods  of  time  or  minor  sections  of  test  are  involved. 

5.  Message  Report.  This  is  the  means  for  reporting  informa¬ 
tion  that  is  of  an  urgent  nature.  It  is  used  for 
reporting  results  of  limited  scope  evaluation  phases  or 
preliminary  summary  results  of  large  scope  evaluations/ 
evaluation  phases. 


Source:  Naval  Air  Test  Center 
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TABLE  III 


1. 


Part  I 


CLASSIFICATION  OF  DEFICIENCIES  BY  THE 
NAVAL  AIR  TEST  CENTER 


a.  Airworthiness  of  the  aircraft. 

b.  The  ability  of  the  aircraft  (or  piece  of  equipment) 
to  accomplish  its  primary  or  secondary  mission 

(or  intended  use) . 

c.  The  effectiveness  of  the  crew  as  an  essential  subsystem. 

d.  The  safety  of  the  crew  or  the  integrity  of  an  essential 
subsystem.  In  this  regard,  a  real  likelihood  of  injury 
or  damage  must  exist.  Remote  possibilities  or 
unlikely  sequences  of  events  shall  not  be  used  as  a 
basis  for  safety  items. 

A  Part  I  deficiency  is  the  most  serious  category  in  which  a 
deficiency  can  be  placed.  It  indicates  an  unsatisfactory 
characteristic.  The  criteria  to  be  used  in  determining 
whether  or  not  a  deficiency  is  a  Part  I  deficiency  is  to  ask 
the  question,  can  the  aircraft  or  piece  of  equipment  accomplish 
its  mission  with  a  satisfactory  degree  of  safety  and  effective¬ 
ness?  If  not,  then  a  Part  I  deficiency  exists. 

2.  Part  II  indicates  a  deficiency  of  lesser  severity  than  a 
Part  I  which  does  not  substantially  reduce  the  ability  of  the 
aircraft  or  piece  of  equipment  to  accomplish  its  primary  or 
secondary  mission,  but  the  correction  of  which  will  result  in 
significant  improvement  in  the  effectiveness,  reliability, 
maintainability,  or  safety  of  the  aircraft  or  equipment.  A 
Part  II  deficiency  is  a  deficiency  which  either  degrades  the 
capabilities  of  the  equipment  or  requires  significant  operator 
compensation  to  achieve  the  desired  level  of  performance; 
however,  the  aircraft  or  equipment  being  tested  is  still 
capable  of  accomplishing  its  mission  with  a  satisfactory 
degree  of  safety  and  effectiveness. 

3.  Part  III  indicates  a  deficiency  which  is  minor  or  slightly 
unpleasant  or  appears  too  impractical  or  uneconomical  to 
correct  in  this  model,  but  should  be  avoided  in  future 
designs . 


Source:  Naval  Air  Test  Center 


categories:  Part  I,  Part  II,  or  Part  III.  Table  III 

outlines  the  criteria  for  deficiency  classification. 

A  Part  I  deficiency,  the  most  serious 
of  the  three  must  be  corrected  as  early  as  "prior  to 
further  flying"  or  as  late  as  "soon  as  possible,  but  not 
to  interfere  with  development."  A  Part  II  deficiency  is  to 
be  corrected  "as  soon  as  practicable."  Lastly,  the  least 
serious  Part  III  deficiency  is  "to  be  avoided  in  future 
designs."  [14:111-27] 

A  breakdown  of  deficiencies  reported 
in  previous  NPEs  and  BIS  trials  pertaining  to  recent  aircraft 
development  programs  is  contained  in  Table  IV.  By  far,  the 
most  frequently  reported  deficiency  is  classified  as  Part  II. 

(3)  Report  Release  and  Distribution.  Release 
and  distribution  of  NAVAIRTESTCEN  and  BIS  reports  is  limited 
to  U.S.  government  agencies  only.  Requests  for  reports  from 
other  interested  parties  must  be  directed  to  either  the 
NAVAIRTESTCEN  or  the  Sub-Board  of  Inspection  and  Survey. 
Normal  distribution  of  reports  will  include: 

--  Cognizant  NAVAIRSYSCOM  Program  Manager 
for  Acquisition  (PMA) ; 

—  Cognizant  NAVAIRSYSCOM  Class  Desk 
Officer; 

—  Cognizant  NAVPRO/DCAS/AFPRO ;  and 

—  Applicable  NAVAIRSYSCOM  Systems 
Engineering  Technical  Division. 

Distribution  of  reports  is  accomplished 
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AIRCRAFT 


Source:  Sub-Board  of  Inspection  and  Survey,  Patuxent  River,  MD 


via  hard-copy.  Only  the  Defense  Documentation  Center  (DDC) 
makes  copy  available  in  microfiche,  and  the  reports  available 
are  limited  to  NAVAIRTESTCEN  DRs ,  TRs ,  and  RTRs . 

c.  Naval  Safety  Center 

While  not  playing  a  major  role  in  volume  of 
experience  feedback,  the  Safety  Center  does  provide  vital 
feedback  related  to  flight  and  maintenance  operations 
safety.  Similar  to  the  reporting  requirements  imposed  upon 
the  commercial  airlines  by  the  Pederal  Avaiation  Administra¬ 
tion,  information  input  into  the  Safety  Center  is  in  the 
form  of  mandatory  reporting  by  fleet  operating  units.  Such 
inputs  include  safety  unsatisfactory  material/condition 
reports,  aircraft  incident  reports,  aircraft  accident 
reports,  and  the  like. 

The  Safety  Center  does  their  own  in-house  data 
compilation,  processing,  and  analysis.  Many  of  the  output 
products  assist  NAVAIRSYSCOM  and  contractor  design  teams  in 
the  elimination  of  undesirable  safety  characteristics  from 
existing  aircraft  systems  and  future  new  development  designs. 

d.  Direct  Reporting  by  NAVAIRSYSCOMHQ 

Yet  another  source  of  external  or  second  level 
information  is  made  available  to  the  prime  contractors. 

The  Program  Management  (PM)  Office  and  the  cognizant  tech¬ 
nical  divisions  are  sources  in  the  NAVAIRSYSCOMHQ  reporting 
directly  to  the  contractor.  See  Figure  14. 


During  the  full  scale  engineering  development 
phase  of  aircraft  weapon  system  acquisition,  several  events 
take  place  under  the  jurisdiction  of  the  PM  that  are  of 
significant  value  to  contractor  design  teams.  These  events 
include  the  full-scale  engineering  mock-up  review,  maintenance 
mock-up  review,  engineering  design  reviews  and  integrated 
logistic  support  management  team  meetings.  The  objectives 
of  these  efforts  are  two-fold:  First,  to  identify  design 
suitability  problem  areas  as  early  as  possible  so  immediate 
corrective  action  will  minimize  the  adverse  impact  of  cost, 
schedule  and  weight;  and  second,  with  the  combined  efforts 
of  Navy  and  contractor  supportability  personnel,  to  develop 
the  best  possible  fleet  deployment  support  package. 

After  the  aircraft  enters  production  and  is 
deployed  with  fleet  operating  units,  a  Naval  Air  Rework 
Facility  (NARF)  is  assigned  as  the  cognizant  Field  Activity 
(CFA)  for  engineering  support  of  the  aircraft.  From  that  time 
on,  until  the  aircraft  is  retired  from  service,  the  NARF  and 
aircraft  contractor  can  be  expected  to  have  intensive  engi¬ 
neering  ties.  Engineering  investigations,  product  improvements 
and  service  life  extension  programs  require  in-depth  engineer¬ 
ing  study.  Engineering  studies  of  this  type  provide  a  wealth 
of  invaluable  information. 

B.  RDTtE  FEEDBACK  LOOP  ACTION  GENERATION  SYSTEM 
1 .  Background 

NAVAIRTESTCEN  Deficiency  Reports  (DR) ,  Board  of 
Inspection  and  Survey  (BIS)  Yellow  Sheets  (Y/S) ,  and  other 
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T&E  reports  discussed  previously  are  the  official  means  by 
which  aircraft  design  deficiencies,  specification  violations 
and/or  failures  in  equipment  are  identified.  Traditionally, 
these  reports  have  been  used  by  cognizant  NAVAIRSYSCOM  and 
contractor  program  management  personnel  for  resolving  serious 
deficiencies  associated  with  the  specific  weapon  system  under¬ 
going  T&E.  The  reported  deficiencies  and  their  solutions  are 
generally  confined  to  the  program  manager  and  his  technical 
staff.  The  "lessons  learned"  are  seldom  applied  to  new 
development  programs  or  made  available  to  industry  design 
teams  as  feedback  for  previous  design  experience. 

Recognizing  the  opportunity  to  expand  the  potential 
utilization  of  T&E  deficiency  data,  the  Air  Crew  Systems 
Division  (AIR-531)  of  NAVAIRSYSCOM  launched  a  program  in  1974 
to  develop  a  mechanized  BIS  Y/S  tracking  system  related  to 
air  crew  aspects  of  aircraft  weapon  systems.  Among  the 
advantages  offered  by  the  system  are:  (13:4] 

—  Greater  consistency  in  the  processing  and 
application  of  Y/S  data; 

—  Prediction  of  potential  problems  in  aircraft 
weapon  systems  entering  development;  and 
—  Identification  of  technology  problems  for  input 
into  the  R&D  and  training  communities. 

A  follow-on  effort,  the  RDT&E  Feedback  Loop  Action 
Generation  System  (FLAGS)  was  undertaken  in  1976  to  expand  and 
refine  the  BIS  Y/S  tracking  system.  As  a  result,  FLAGS  is 
capable  of  providing  deficiency  status  reports,  narrative 
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summary  reports,  statistical  compilations,  and  the  correlation 
of  problem  areas  common  to  all  fixed  and  rotary  wing  aircraft 
weapon  systems. 

2 .  System  Objectives 

FLAGS  has  been  designed  to  facilitate  the  storage  and 
processing  of  statistical  (quantitative)  and  narrative  (quali¬ 
tative)  information.  Most  important  in  this  area  is  the  sys¬ 
tem's  capability  to  store  and  process  aircraft  system,  sub¬ 
system,  and  component  deficiency  data  resulting  from  mock-up 
reviews,  design  reviews,  maintenance  mock-up  reviews,  Navy 
Preliminary  Evaluation  (NPE) ,  Board  of  Inspection  and  Survey 
(BIS)  trials  and  Service  Life  Extension  Program  (SLEP) 
engineering  investigations. 

The  aggregate  of  these  individual  technical  informa¬ 
tions  under  one  system  (a  NAVAIR  corporate  memory  for  previous 
design  experience)  and  the  ability  to  easily  retrieve  user 
information,  has  satisfied  most  of  the  FLAGS  objectives. 

Those  objectives  thought  to  have  significance  related  to 
identifying  and  applying  previous  design  experience  are 
listed  in  Table  V. 

3 .  Information  Retrieval 

Retrieval  of  information  from  the  FLAGS  data  base  is 
limited  only  by  the  amount  and  the  type  of  information  stored 
in  the  data  base.  A  simplified  flow  from  user  data  query  to 
output  product  is  shown  in  Figure  15. 

Information  retrieval  is  initiated  by  completing  a 
retrieval  request  form  using  applicable  FLAGS  coding  sheets 
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TABLE  V 


OBJECTIVES  OF  THE  FEEDBACK  LOOP 
ACTION  GENERATION  SYSTEM  (FLAGS) 


1.  Providing  rapid  retrieval  of  NPE/BIS  deficiency  data  in 
simple  format. 

2.  Printing  out  selective,  detailed  deficiency  reports  to 
facilitate  specific  management  decisions. 

3.  Indicating  "avoid  in  future  design"  deficiencies. 

4.  Associating  T&E  deficiency  data  rapidly  with  Mil-Specs/ 
Stds  for  updates  and  follow-on  buys. 

4.  Identifying  complex,  recurring  deficiencies  for  research 
and  technology  assistance. 

6.  Contributing  to  R&D  requirements  definition. 

7.  Permitting  statistical  manipulations  across  deficiency 
base  to  compile  summary  statistics  on  critical  issues. 

8.  Enabling  flexible  access  to  technical  data  and  informa¬ 
tion  across  past  weapon  system  deficiencies  for 
correlation  with  new  deficiencies. 

9.  Relating  system  development  data  (program,  design, 
mock-up  decisions)  to  contractor  demo  data,  NPE,  BIS, 
OPEVAL  and  fleet  gripes. 

10.  Providing  research  by  specific  technical  keywords, 

acronyms,  work  unit-codes  and  unsatisfactory  reports. 


SOURCE:  NAVAL  AIR  SYSTEMS  COMMAND  (AIR-531) 
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SIMPLIFIED  SYSTEM  FLOV' 


Source:  Naval  Air  Systems  Command,  FLAGS 

FIGURE  15:  FLAG  SYSTEM  —  SIMPLIFIED  PROCESS  FLOW 


as  a  guide  to  the  retrieval  criteria.  The  request  is  then 
processed  by  the  NAVAIRSYSCOM  computer  facility.  Retrieval 
results  in  a  computer  printout  tailored  to  the  specific 
requirements  on  the  retrieval  request  form.  Figure  16 
illustrates  a  typical  user  query  and  output  product. 

More  detailed  examples  of  retrieval  requests  and 
output  products  are  found  in  Appendix  E.  The  many  advantages 
of  FLAGS  are  found  in  Table  VI . 

C.  GOVERNMENT-INDUSTRY  DATA  EXCHANGE  PROGRAM 
1.  Background 

The  Government- Industry  Data  Exchange  Program  (GIDEP) 
chartered  in  1970  became  the  scheme  to  consolidate  the  then 
existing  Tri-Service  Interagency  Data  Exchange  Program  (IDEP) 
Under  this  new  scheme,  the  Chief  of  Naval  Material  was  desig¬ 
nated  by  the  Joint  Logistics  Commanders  to  exercise  over-all 
responsibility  for  GIDEP. 

Essentially,  GIDEP  is  a  cooperative  activity  between 
government  and  industry  participants  seeking  to  conserve 
time,  personnel  and  money  by  making  maximum  use  of  existing 
experience  and  knowledge.  The  program  provides  a  means  to 
automatically  exchange  a  multitude  of  different  types  of 
technical  information  essential  in  the  research,  design, 
development,  T&E,  production  and  operational  phases  of  the 
life  cycle  of  aircraft  weapon  systems.  [9:1-1]  To  accom¬ 
plish  this  task,  GIDEP  is  concerned  with  the  acquisition, 
storage,  retrieval  and  dissemination  of  technical  information 
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DISK  STORAGE 


ADVANTAGES  OF  SYSTEM 


2 .  Program  Objectives  and  Policy 

Identified  GIDEP  objectives  relevant  to  design  are 

listed  in  Table  VII. 

All  activities  of  the  Naval  Material  Command  directly 
engaged  in  the  research,  design,  development,  testing  and 
production  of  mission-related  Navy  material  are  required  to 
participate  in  GIDEP.  Furthermore,  all  Navy-funded  contracts 
for  research,  design,  development,  testing,  and  production 
of  mission-related  Navy  material  wherein  the  contract  amount 
exceeds  $100,000,  shall  require  contractor  participation 
in  the  program.  [15:2] 

3 .  Program  Operation 

GIDEP  emphasis  has  been  placed  upon  the  rapid  trans¬ 
mission  of  information  to  potential  users,  and  upon  having 
the  information  rapidly  available  upon  demand.  A  highly 
simplified  illustration  of  input  and  output  information  flow 
is  found  in  Figure  17.  A  data  retrieval  system  developed  by 
GIDEP  allows  rapid  accessibility  to  microfilmed  information 
in  the  data  bank,  either  through  hard  copy  indexes  or  by  access 
to  the  GIDEP  operation  center's  computer  data  system  via  a 
remote  computer  terminal. 

GIDEP  places  primary  emphasis  on  data  generated  by 
users  rather  than  contractors.  Government  specifications, 
contractor  proprietary  data  and  classified  information  are 
not  within  the  scope  of  the  current  program. 
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TABLE  VII 


OBJECTIVES  OF  THE  GOVERNMENT- 
INDUSTRY  DATA  EXCHANGE  PROGRAM  (GIDEP) 


1.  Reduce  or  eliminate  duplicative  expenditures  for  the 
testing  of  parts,  components  and  materials. 

2.  Increase  system  reliability. 

3.  Provide  advance  notification  of  possible  part, 
component  and  material  failures  or  potential 
problem  areas. 

4.  Promote  standard  procedures  for  reporting  test 
information. 

5.  Facilitate  communication  among  scientific  and 
technical  personnel  working  on  related  programs. 

6.  Provide  a  source  of  general  parts,  components  and 
materials  test  data  during  research,  development 
and  other  stages  of  the  procurement  cycle. 

7.  Provide  failure  rate  and  mode  data  to  assist  in 
design,  reliability  and  logistics  functions. 

8.  Provide  a  failure  experience  data  bank  on  parts, 
components  and  materials  to  identify  defective/ 
suspect  items,  facilitate  their  removal  from 
stock  and  preclude  their  reentry  into  new  equipment 
designs. 

9.  Provide  for  the  transfer  of  technology  to  American 
industry  in  the  interest  of  the  national  economy. 


SOURCE:  Headquarters,  NAVAL  MATERIAL  COMMAND  (MAT-06B) 


84 


GIDEP  SYSTEM  INFORMATION  FLOW  Code  805-Sep  77 


4 .  Data  Banks 


Participants  in  the  program  are  provided  access  to 
four  major  data  banks.  [9:1-5] 

a.  Engineering  Data  Bank 

This  bank  contains  engineering  evaluation  and 
qualification  tests  reports,  nonstandard  parts  justification 
data,  parts  and  materials  specifications,  manufacturing  pro¬ 
cesses,  failure  analysis  data,  and  other  related  engineering 
data  on  parts,  components,  materials,  and  processes. 

b.  Metrology  Data  Bank 

This  bank  contains  related  metrology  engineering 
data  on  test  systems,  calibration  systems,  and  measurement 
technology  and  test  equipment  calibration  procedures,  and 
has  been  designated  as  a  data  repository  for  the  National 
Bureau  of  Standards  (NBS !  metrology  related  data. 

c.  Failure  Experience  Data  Bank 

This  bank  contains  objective  failure  information 
generated  vhen  significant  problems  are  identified  on  parts, 
components,  processses,  fluids,  materials,  or  safety  and 
fire  hazards.  This  data  bank  includes  the  ALERT  and  SAF- 
ALERT  system  and  failure  analysis  information. 

d.  Reliability  and  Maintainability  Data  Bank 

This  bank  contains  failure  rate/mode  and  replace¬ 
ment  rate  data  on  parts  and  components  based  on  field  per¬ 
formance  information  and/or  reliability  demonstration  tests 
of  equipment,  subsystems  and  systems.  The  data  bank  also 
contains  reports  on  theories,  methods,  techniques,  and 
procedures  related  to  reliability  and  maintainability  practices. 
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5.  Information  Utilization 

Participation  in  GIDEP  has  shown  steady  growth  in 
recent  years.  The  total  number  of  participating  government 
and  industry  activities  has  grown  from  407  in  1974  to  554  in 
1978.  Documentation  activity  has  shown  a  five-fold  increase. 
Documents  received  (input)  increased  from  3,325  in  1974 
to  15,257  in  1978,  and  documents  processed  increased  from 
3,301  in  1974  to  17,452  in  1978.  [7]  Design  engineers, 

reliability  and  maintainability  engineers,  and  logisticians 
alike,  are  utilizing  GIDEP  information  to  expedite  and  improve 
their  decision-making  process.  Forms  of  information,  groups 
utilizing  the  information  and  the  perceived  value  of  such 
information  is  summarized  in  Table  VIII. 
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TABLE  VIII 
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TABLE  VIII  (Continued) 
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VI.  SUMMARY  OF  FINDINGS 


Previous  sections  of  this  study  have  explored  the  various 
sources  of  operational  experience  information  in  commercial 
and  military  aircraft  programs.  These  same  sections  also 
addressed  the  methods  employed  in  the  flow  of  such  informa¬ 
tion  back  to  the  contractor  and  his  product  support  and 
design  engineering  personnel.  This  section  summarizes  the 
more  important  findings  made  during  study  of  the  commercial 
and  military  feedback  process. 

A.  MOTIVATION  FOR  FEEDBACK 

The  most  apparent  difference  between  commercial  and  mili¬ 
tary  programs  for  operational  experience  feedback  is  economic 
motivation.  Economic  motivation  provides  catalytic  action 
for  commercial  program  but  is  not  found  as  critical  within 
military  programs.  To  remain  competitive  in  the  commercial 
marketplace  for  additional  sales  of  an  existing  aircraft  model, 
and  to  achieve  a  favorable  reputation  for  future  sales  of  a 
new  development  aircraft,  contractors  rely  heavily  on  "lessons 
learned"  from  operational  experience.  These  lessons  have  been 
expressed  and  substantiated  in  terms  of  cost  effectiveness 
by  airlines  for  a  long  time.  Military  services  have  been 
fully  aware  of  life-cycle  costs  only  for  a  short  time  and  the 
available  data  base  is  small  by  comparison  with  the  air 
transport  industry. 


Navy  Acquisition  and  Fleet  operating  practices  too 
often  preclude  the  contractor  from  adequately  monitoring 
his  deployed  products.  The  vast  majority  of  experience 
feedback  to  the  contractor  is  via  a  feedback  system  external  to 
his  organization.  Many  specific  data  needs  desired  by  the 
contractor  are  not  obtainable  from  Navy  sources.  This  will 
necessarily  influence  the  contractor's  motivation  regarding 
feedback. 

Navy  and  contractor  information  feedback  systems  differ 
in  philosophy.  The  latter  recognizes  at  all  corporate 
management  levels  that  dispatch  reliability  is  influenced  by 
operations  and  environment;  that  it  is  essential  to  monitor 
operational  reliability.  Consequently,  contractor  Technical 
Management  Information  Systems  (TMIS)  or  operational  support 
system  data  bases  are  drive  by  reliability  programs.  These 
programs  in  turn,  drive  technical  advancement  programs  for 
product  improvement  and  new  design  development. 

The  principal  Navy  information  feedback  system,  its 
Maintenance,  Material  and  Manpower  Management  System  (3-M) 
was  not  conceived  as  a  reliability  monitoring  system.  Rather, 
the  3-M  system  was  conceived  as  a  manpower  and  material 
management  system.  Since  1965,  the  3-M  system  has  gradually 
attained  a  reliability  orientation;  however,  pending  a  change 
in  policy  and  system  design,  3-M  remains  primarily  a  Main¬ 
tenance,  Material  and  Manpower  Management  feedback  system. 

Contractor  TMIS  and  the  Navy's  3-M  system  have  evolved 


as  mechanized  systems  to  provide  for  mass-processing  of 


generated  data  into  usable  information.  Nevertheless,  both 
systems  exhibit  definite  differences  attributable  to  feedback 
motivation  differences.  Table  IX  summarizes  the  major 
differences . 

B.  ACCESS  TO  FEEDBACK  INFORMATION 

Contractor  access  to  the  numerous  sources  of  feedback 
information  varies  widely.  Access  to  information  generated 
external  to  the  contractor's  organization  is  often  difficult 
if  not  impossible.  The  weakest  link  in  the  external  informa¬ 
tion  feedback  loop  is  in  the  area  of  qualitative  design- 
oriented  experience.  The  reasons  appear  to  be  a  combination 
of  two  factors  —  the  proprietary  nature  and  the  fragmented 
control  of  such  information. 

1.  Proprietary  Information 

Experience  evolving  from  full-scale  engineering  develop¬ 
ment  supplements  experience  subsequently  obtained  during  ser¬ 
vice  use  by  operating  activities.  Information  and  experi¬ 
ence  generated  during  development  is  of  immense  value  to 
aircraft  design  teams  in  that  it  serves  not  only  in  making 
early  corrections  to  existing  system  discrepancies  but  more 
importantly  in  providing  "lessons  learned"  to  avoid  identical 
or  similar  problems  from  reoccurring  in  subsequent  design 
efforts. 

Navy  acquisition  policies  and  practices  honor  develop¬ 
ment  experience  generated  in-house  as  proprietary  information. 
Such  information  includes  mock-up  review  chits,  design  review 
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PRINCIPAL  OPERATIONAL  EXPERIENCE  FEEDBACK 


Data  limited  to  corporate  products  -  Data  includes  all  inventory/type/ 

model  series  aircraft  since  1965 


actions,  and  test  and  evaluation  results.  Development 
information  access  is  limited  to  U.S.  government  agency 
and  cognizant  development  contractor  use  only.  Access 
or  transfer  of  this  information  to  other  contractors 
within  the  aerospace  industry  is  not  practiced. 

The  industry  faces  a  similar  situation  in  commer¬ 
cial  aircraft  development  programs.  The  intense  competi¬ 
tive  nature  for  aircraft  sales  to  the  world's  airlines  inhibits 
transfer  of  current  and  previous  experience  among  competing 
contractors . 

2 .  Fragmented  Sources 

Major  manufacturers  of  transport  aircraft  have  each 
developed  a  corporate  memory  regarding  their  previous  pro¬ 
ducts.  Statistical  (quantitative)  and  narrative  (quali¬ 
tative)  information  is  normally  contained  in  a  technical 
management  information  system  (TMIS)  operated  by  either  a 
commercial  product  support  or  a  corporate  experience  reten¬ 
tion  group.  Centralized  information  control  is  the  key 
for  ready  access. 

For  Navy  programs,  integration  efforts  directed  at 
developing  a  centralized  data  bank  for  Navy-wide  design  and 
operational  experience  have  been  minimal.  Instead,  a  number 
of  individually  sponsored  feedback  systems  serving  the 
specific  needs  of  the  sponsor  have  sprouted  during  recent 
years. 


A  contractor  must  now  weave  his  way  through  a  multi¬ 
tude  of  fragmented  Navy  sources  to  recover  development,  test 
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and  evaluation,  and  fleet  operation  experience.  Recovery 
efforts  from  this  extensive  and  fragmented  data  base  are 
often  time  consuming  and  usually  result  in  information  that 
is  difficult  to  correlate  into  terms  meaningful  to  contractor 
design  engineering  teams. 

As  shown  in  Section  V,  B  and  C,  FLAGS  and  GIDEP  have 
been  introduced  by  the  Navy  to  provide  narrative  or  quali¬ 
tative  experience  information  (i.e.,  T&E  report  results,  design 
review  chits,  mock-up  review  chits,  etc.).  They  complement 
the  3-M  system  which  provides  operational  statistical  or 
quantitative  data  from  operational  units  as  described  in 
Section  V.A.2.  Basically,  FLAGS  and  GIDEP  systematize 
existing  Navy  data  sources  into  a  computerized  data  and  informa¬ 
tion  base  peculiar  to  the  needs  of  the  two  systems.  Entry 
into  these  storage  and  feedback  information  systems  is  accom¬ 
plished  via  specific  technical  keywords,  acronyms,  and 
work  unit  codes  that  are  kin  to  the  aerospace  design  engin¬ 
eering  community.  Entry  is  limited  only  by  the  degree  of 
system  software  flexibility. 

Each  of  the  three  feedback  systems  is  under  separate 
sponsorship  and  funding  —  FLAGS  with  the  Naval  Air  Systems 
Command,  GIDEP  with  the  Naval  Material  Command,  and  3-M 
with  the  Chief  of  Naval  Operations. 

C.  QUICKNESS  OF  INFORMATION  FEEDBACK  AND  RETRIEVAL 

Information  on  operational  experience  must  be  quickly 
retrievable  to  be  of  value.  The  air  transport  industry  has 
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long  recognized  the  importance  of  expediency  in  feedback  of 
operational  experience  to  the  manufacturers  for  correction 
of  flight  equipment  problems.  Management  emphasis  on  expedi¬ 
ency  is  evident  by  world-wide  real-time  communication  networks 
with  field  feedback  sources.  Operations  of  this  type  are 
for  the  most  part,  indirectly  financed  by  the  airline 
customers  who  find  such  investments  highly  cost-effective 
in  reducing  lost  revenue  caused  by  delayed  or  grounded 
aircraft. 

Navy  feedback  sources  are  less  responsive.  Often,  during 
the  dynamics  of  full-scale  engineering  development,  Navy  T&E 
feedback  is  too  late  to  affect  the  program's  original  design 
decision.  Also,  the  receipt  of  Navy  T&E  results  by  the  pro¬ 
curing  activity  has  the  effect  of  seeming  to  put  NAVAIRSYSCOM 
personnel  "on  report."  A  defensive  attitude  on  the  part  of 
the  program  manager  and  his  technical  team  inevitably  results. 
Instead  of  being  a  tool  for  the  acquisition  and  fielding  of 
the  best  aircraft  weapons  system  possible,  T&E  feedback  tends 
to  embarrass  the  program  management  and  technical  team, 
thereby,  suppressing  feedback  of  badly  needed  design  experi¬ 
ence  to  contractor  design  teams. 

Utilization  of  information  on  operational  experience  by 
designers  during  the  design  process  on  new  development  pro¬ 
grams  is  time-critical.  To  be  of  value,  designers  must  have 
the  information  when  needed  —  preferably  via  a  remote  CRT 
terminal  and  within  easy  reach. 
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FLAGS  and  GIDEP  have  the  capability  of  improving  the 
situation  in  favor  of  more  responsive  feedback  and  information 
retrieval.  Both  systems  mechanize  the  data  base  which  in 
today's  environment  has  grown  so  that  it  is  too  extensive  for 
unaided  manual  handling.  Data  inputs  come  directly  from  the 
source,  thereby  streamlining  the  existing  manual  process  and 
minimizing  feedback  delays.  Output  products  are  available 
in  both  hard  copy  or  through  on-location  CRT  terminals. 

D.  APPLICATION  OF  FEEDBACK  INFORMATION 

In  commercial  aircraft  programs,  operational  experience 
is  greatly  emphasized  and  a  centralized  feedback  system  has 
been  developed.  Information  revealing  product  deficiencies 
is  immediately  applied  to  current  programs  for  product 
improvement.  Equally  important,  deficiencies  serve  as 
"lessons  learned"  for  avoiding  recurring  or  similar  problems 
in  new  development  programs.  Experience  with  enhancing 
design  features  from  past  products  is  also  applied  to  new 
development  programs.  Such  features  can  substantially  con¬ 
tribute  to  improved  performance  and  effectiveness  compared  to 
previous  products. 

In  Navy  programs,  with  the  exception  of  3-M  data,  feedback 
is  predominantly  applied  to  solving  the  most  visible  aspects 
of  the  "now"  problems  associated  with  individual  aircraft  or 
equipment  subsystems.  Little  incentive  exists  to  apply  this 
feedback  information  to  new  development  programs,  partly  due 
to  different  priorities  for  Navy  program  managers,  and  partly 


due  to  Navy  acquisition  policy  and  practices  which  seldom 
provide  the  necessary  funding. 

The  Navy  3-M  data  bank  is  made  available  to  all  defense 
contractors.  Currently,  it  provides  only  statistical  (quan¬ 
titative)  information  pertaining  to  aeronautical  maintenance, 
material,  and  manpower  on  all  Navy  aircraft.  Extensive 
application  for  3-M  is  in  the  area  of  systems  effectiveness. 
Contractors  rely  on  3-M  data  for  system  reliability  and 
maintainability  prediction  and  allocation. 

FLAGS  and  GIDEP  have  both  been  experiencing  recent  growth 
in  utilization.  The  founder  of  FLAGS,  the  Aircrew  Systems 
Division  (AIR-531)  of  the  NAVAIRSYSCOM,  has  also  been  its 
chief  user.  Since  its  inception  in  the  mid  1970s,  FLAGS  has 
been  employed  as  a  means  for  tracking  air  crew  systems 
deficiencies  in  Naval  aircraft  development  programs. 

Recently,  FLAGS  has  found  application  to  the  F-18  air¬ 
craft  weapon  system  development  program.  In  January  of 
1978,  the  NAVAIRTESTCEN  highlighted  the  need  for  an  automated 
system  of  tracking  F-18  deficiency  reports.  The  intent  of 
FLAGS  application  in  this  program  as  discussed  in  Appendix 
F,  is  to  derive  better  administrative  control  and  manage¬ 
ment  action  of  deficiency  reports.  Once  again,  feedback  empha¬ 
sis  is  being  directed  at  the  immediate  need  for  improving 
program  administrative  and  management  functions.  Little 
emphasis  has  been  directed  at  building  an  experience  infor¬ 
mation  base  for  utilization  by  design  teams  in  avoiding 
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similar  or  recurring  problems  in  future  Naval  aircraft 
development  programs . 

As  previously  discussed  in  Section  V,  GIDEP  utiliza¬ 
tion  and  fields  of  application  have  increased  markedly 
since  1974.  Breakdown  of  GIDEP  utilization  is  roughly  75 
percent-  industry  and  25  percent  government.  Payoff  for  users 
of  GIDEP  is  estimated  at  fourteen  dollars  recovered  for 
every  dollar  spent.  Despite  recent  growth  in  the  number  of 
participants  and  frequency  of  activity  by  both  the  industry 
and  the  government,  the  NAVAIRSYSCOM  and  its  RDT&E  field 
activities  have  taken  little  advantage  of  the  services  and 
wealth  of  technical  experience  information  presently  avail¬ 
able.  [8:35] 
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APPENDIX  A 


ORGANIZATIONS  CONTACTED  AND  INTERVIEWED 

Organizations  contacted  and  interviewed  during  the 
course  of  this  study  include  the  following: 

DEPARTMENT  OF  THE  NAVY 

Navy  Maintt  mce  Support  Office 

Government- Indus try  Data  Exchange  Program,  Operations 
Center 

Naval  Air  Rework  Facility,  North  Island 

Naval  Air  Systems  Command 

Naval  Air  Test  Center 

Naval  Aviation  Logistics  Center 

Naval  Material  Command 

Sub-Board  of  Inspection  and  Survey,  Patuxent  River 
AEROSPACE  INDUSTRY 

Air  Transport  Association  of  America 
Boeing  Commercial  Airplane  Company 
General  Electric  Aircraft  Small  Engine  Group 
Lockheed-California  Company 
Piedmont  Airlines 
Vought  Corporation 
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APPENDIX  B 

LOCKHEED  -  CALIFORNIA  COMPANY 
A  Division  of  Locfchssd  Aircraft  Corporation 

MANAGEMENT  DIRECTIVE 

COMMERCIAL  PRODUCT  RELIABILITY 
AND  MAINTAINABILITY 


NUMMR:  1 62 

FIRST  MSUCO:  10-  7-76 
RSVISCD: 


L  It  is  the  policy  of  the  Lockheed- California  Company  to 


A.  Achieve  a  realistically  high  level  of  dispatch  reliability  and  utilization 
for  its  products,  and  to  that  end 


1.  Establish  standards  and  specifications  for  reliability  and 
maintainability  commensurate  with  program  needs  and  compatible 
with  customer  requirements. 

2.  Establish  plans,  programs  and  controls  to  achieve  and  sustain 
required  levels  of  reliability  and  maintainability  during  develop¬ 
ment  and  production  programs  and  during  operational  life. 

3.  Establish  standards  and  techniques  for  Failure  Mode  and  Effects 
Analysis  (FMEA)  and  for  Quantitative  Fault  Tree  Analysis 
(QFTA)  commensurate  with  the  requirement  of  governmental 
regulatory  agencies. 

4.  Maintain  a  data  collection  and  feedback  system  to  provide 
information  useful  in  identifying  and  solving  reliability  and 
maintainability  problems  and  establishing  risk  levels. 

II.  Responsibility  is  assigned  and  authority  granted  to 
A.  Commercial  Product  Support  Branch  to 


1.  Establish  quantitative  reliability  and  maintainability  levels  in 
proposals,  contracts  and  specifications. 

2.  Establish  mathematical  models  and  techniques  for  computing 
maintenance  costs,  on-aircraft  corrective  maintenance  time 
allocations,  reliability  levels,  dispatch  levels  and  probability 
of  failure  allocation. 

3.  Develop  engineering  maintainability  and  reliability  design 
criteria  and  conduct  design  surveillance.  Prepare  maintenance 
engineering  analyses  and  reliability  analyses. 

4.  Prepare  allocation  of  quantitative  maintainability  and  reliability 
levels  which  are  to  be  reflected  in  the  program  plan. 

5.  Establish  reliability  and  maintainability  test  requirements  for 
components  and  sytems  as  necessary  to  perform  Quantitative 
Fault  Tree  Analyses,  Failure  Mode  and  Effects  Analyses, 
Maintenance  downtime  assessment,  and  to  establish  Trouble 
Shooting  Procedures. 
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IL  A.  6. 

7. 

8. 

9. 

10. 


11. 

12. 

13. 

14. 


Evaluate  progress  of  operational  programs  and  report  to  management 
concerning  achievement  of  maintainability  levels,  reliability  levels, 
and  probability  of  failure  allocations. 

Establish  requirements  for  piece-part  screening  and  burn- in/ run- in 
in  specifications  and  proposals.  Monitor  these  parameters  in 
operational  programs  as  necessary  to  achieve  the  required  reliability 
levels,  dispatch  levels  and  probability  of  failure  allocations. 

Provide  technical  direction  over  the  execution  of  initial  measurement 
programs  and  participate  in  all  commercial  reliability  and  maintain¬ 
ability  measurement  programs. 

Identify  and  analyze  problem  areas  affecting  achievement  of  maintain¬ 
ability  and  reliability  goals  and  work  with  cognizant  organizations  to 
establish  corrective  action. 

In  coordination  with  other  branches,  negotiate  and  maintain  liaison 
with; 

a.  LCC  customers  regarding  achievement  of  reliability  and 
maintainability  goals. 

b.  LCC  suppliers  regarding  achievement  of  reliability  and  maintain¬ 
ability  requirements,  and  compliance  with  their  R&M  program  plans. 

c.  Regulatory  agencies  regarding  maintenance  program  development 
and  revisions,  equipment  reliability  performance,  and  achievement 
of  probability- of- failure  levels. 

Prepare  maintenance  and  facilities  planning  documents  conforming  to 
maintainability  plans  and  requirements.  Coordinate  with  Engineering 
in  the  resolution  of  any  problems  arising  from  engineering  plans  and 
requirements. 

Collect  and  provide  data  from  field  operations  and  field  flight  test 
programs  to  establish  equipment  reliability  level. 

Establish  product  reliability  and  maintainability  data  needs  for 
inclusion  in  the  integrated  failure-data  collection  and  feedback  systems. 
Participate  in  the  development  of  data  collection  program  plans  and  in 
the  application  of  automatic  data  processing  techniques. 

Provide  the  Engineering  Branch  with  M&R  assistance  in  the  design  of 
each  Calac  commercial  product  so  that  it  incorporates  requirements 
for  ' naintainability,  reliability  and  supportability  consistent  with 
those  for  performance,  initial  and  operational  cost,  producibility, 
and  weight  and  stress. 
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A.  15.  Perform  Calac  coordinating  efforts  for  participation  in  industry  and 

government  activities  affecting  maintainability  and  reliability 
techniques.  Maintain  liaison  with  Lockheed  companies,  as  appro¬ 
priate,  to  insure  timely  interchange  of  maintainability,  reliability 
and  risk  measurement  and  prediction  techniques. 

B.  Engineering  Branch  to 

1.  Design  each  Calac  product  so  that  requirements  for  maintainability, 
reliability,  and  supportability  are  given  consideration  equal  to 
those  for  performance,  initial  and  operational  cost,  producibility, 
and  weight  and  stress. 

2.  Assure  that  all  data  and  information  on  reliability  and  maintainability 
problems  which  become  evident  during  development  test  programs  are 
made  available  to  cognizant  design,  reliability  and  maintainability 
personnel. 

C.  Quality  Assurance  Branch  to 

1.  Participate  with  the  Commercial  Product  Support  Branch  in  estab¬ 
lishing  programs  and  controls  to  verify  conformance  of  products 
and  components  to  reliability  and  maintainability  requirements. 

2.  Establish  and  maintain  an  in-plant  data  center  and  acquire  in-plant 
production  failure  data  sufficient  to  support  reliability  and  main¬ 
tainability  activities. 

3.  Maintain  liaison  with  suppliers  in  coordination  with  Materiel  and 
Commercial  Product  Support. 

4.  Coordinate  with  responsible  Branches  in  the  identification  of  design 

problems  and  in  the  analysis  of  failure. 

5.  Participate  with  the  responsible  Branch  in  the  conduct  of  reliability 
and  maintainability  measurement  programs  for  in-plant  activities. 

D.  Materiel  Branch  to 

1.  Negotiate  supplier  guarantees  for  reliability  and  maintainability  on 
selected  items  as  required  to  satisfy  overall  system  requirements. 

2.  Participate  with  Quality  Assurance  and  Commercial  Product  Support 
in  the  negotiation  for  correction  of  quality  and/or  design  deficiencies 
necessary  to  achieve  required  reliability  and  maintainability  levels. 
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n. 


E.  Administrative  Sc  Computer  Services  Branch  to 

1.  Develop,  at  request  of  using  Branches,  automatic  data  processing 
programs  to  support  the  reliability  and  maintainability  data  collection 
and  feedback  system. 

2.  Provide  automatic  data  processing  reports  to  affected  organizations 
according  to  programmed  requirements. 

F.  Commercial  Program  Planning  to 

1.  Issue  program  letters  when  required  to  coordinate  interbranch 
activities  associated  with  reliability  and  maintainability  plans. 

Establish  and  monitor  schedules. 

G.  All  affected  organizations  to 

1.  Participate  in  the  collection  and  evaluation  of  reliability  and  maintain¬ 
ability  data  when  requested  to  do  so  by  the  cognizant  branches. 

Explanatory  Remarks 

1.  Management  Directive  40  establishes  the  Calac  policy  to  ensure  that  Calac 
products  meet  all  requisite  standards  of  quality. 

2.  Management  Directive  85  establishes  the  Calac  policy  to  provide  support 
equipment  and  related  data  required  for  the  efficient  operation  and 
maintenance  of  each  Calac  product. 

3.  Management  Directive  143  establishes  the  Calac  policy  to  provide  service 
support  required  for  the  proper  operation,  maintenance,  and  best  use  of 
each  Calac  product. 

4.  The  term  "Reliability"  includes  the  establishment  and  measurement  of 
the  product  risk  level,  failure  mode  and  effects  analysis,  and  quantitative 
fault  tree  analysis. 

5.  "Risk"  is  defined  as  the  quantitative  probability  of  a  flight  critical  failure 
mode  occurring  within  a  specific  interval. 


M-t 

D.  O.  Wood 
President 

Lockheed- California  Company 
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APPENDIX  C 

SAMPLES  OF  OUTPUT  REPORTS  FROM 
LOCKHEED'S  OPERATIONAL  SUPPORT  DATA  SYSTEM 

The  following  reports  are  a  sampling  of  the  type  and 
variety  of  reports  periodically  processed  by  the  L-1011 
TriStar  OSDS  program.  These  standard  output,  yet  specific 
reports,  are  considered  pertinent  to  the  area  of  previous 
design  experience  feedback. 

Dispatch  Reliability  -  Fleet  (p.106) 

Mechanical  Systems  -  Dispatch  Reliability  (p.107) 
Landing  Gear  Dispatch  Reliability  (p.  108-109) 

Flight  Control  Dispatch  Reliability  (p.110-111) 
Component  Reliability  Improvement  Program  (p.112-113) 


SOURCE:  Lockheed-Calif ornia  Company 
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DISPATCH  RELIABILITY-FLEET 


MONTHS  AFTER  START  OF  AIRLINE  SERVICE 
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S/B  32-138  will  be  issued  ECD  week  of  1  29  79  to  authorize 
the  Phase  II  system. 


ATA  27  FLIGHT  CONTROLS 
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APPENDIX  D 


AVIATION  RECURRING  AND  ON  DEMAND 
INFORMATION  REPORTS  PRODUCED  BY 
NAMSO 


Aviation  Information  Reports  produced  by  the  Navy 
Maintenance  Support  Office  and  available  to  both  Department 
of  Defense  activities  and  Defense  contractors  are  listed 
on  the  following  pages  of  this  appendix. 


SOURCE:  Navy  Maintenance  Support  Office 
Instruction,  NAMSOINST  4790.1, 
Dated  1  December  1978 


PART  II  -  3-M  AVIATION  INFORMATION  REPORTS  LISTED  IN  REPORT  NUMBER  SEQUENCE 
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APPENDIX  E 


SAMPLES  OF  FEEDBACK  LOOP  ACTION 
GENERATION  SYSTEM  (FLAGS)  RETRIEVAL 
REQUESTS  AND  OUTPUT  PRODUCTS 

The  following  three  examples,  consisting  of  retrieval 
request  followed  by  output  product,  exhibit  the  inherent 
flexibility  of  FLAGS  to  analyze,  correlate  and  retrieve 
information  of  different  types  and  format. 


Example  1  -  Deficiency  Status  Summary  (p.119-120) 

Example  2  -  BIS  Y/S  Deficiency  Identification  (p. 121  -122) 
Example  3  -  Detailed  Deficiency  Report  (p.  123-124) 


SOURCE:  Naval  Air  Systems  Command 


Datc/Tiinc  In:  _FV_  Fct>-  _ OQO£> 

Dot  c/Timc  fl(.r/ui<ed:  _  3.  If?.....  \A_QO 

Dute/Time  Out: _ _ _ _ 


Request  Omjiiiatuf  (Nuinu,  CotU:,  RliJy/Rm,  Telephone): 

j.  Uemln^uja^  }  si\t  SP-< a./ma,  ucUL''V+97 


Antroll  _  .  .  ,  a. 
Type'  F“\HH 


Dc-(i«.i.  i  icy 
Identification _ 


Statement  of  Retrieval  Requirement 


Narrative  r\  _  _  It 

_ c.iteyory_inS_  'T>C€  ‘'pRin  .out 

^hfRi-na. 


U.-p  - ■ 4o  -  cta-k:  FD  d)  i  c.\en  c  u  v^ksi-u-s  vS u-m  u 

Ws+xncj  o^a  ail  F'i4Y\  x*XS  ^elNouo  vShecT 

dLef  \C\Cn  OCS  . 


Printout  Format: 

Deficiency  Status  Summary: 
Detailed  Deficiency  Repoit: 

Statistical  Compilation  _ 

Description  as  requited: 


RETRIEVAL  REQUEST  (EXAMPLE  -  I) 
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Datc/Time  In:  ^  jlL  b.  4  4? _ O  5  3  O 

Date/Time  Required:  _^2  1  4  I 4r  I  CCO 

Dale/Time  Out: _ 


Request  0 r iijin.it or  (Name,  Code.  UUIy/Rm,  1  eleplione):  , 

^).  Uo  nuAC^uod Lj  _  3  .51  ,  JP-.2  / li  “4  8, 


^12-  1H8  7 


Airci iift  .-1,1,4  Deficiency  Narrative  .  _ 

Type: _ IL! _  Identification.. _  Category  r I S  '  |  X'  K  ) 2 > j  1 1  1  ‘  C 

Statement  of  Retrieval  nequiiciTient: 

l.\S^\’AQ  oj-  c4  I  I  1  t'40  1 1 c  lO  .shft'i  u\h 

d  SS  I  C?  f\Lci  V  t  Lj  U_'C"i<  CU6  f 

HIcjKI  »Xl  ncuvt  CS  ,  CdTTRlCC  ItHOcl'.ACys 


Printout  Format: 

Deficiency  Status  Summary: 
Detailed  Deficiency  Report:. 

Statistical  Compilation: _ 

Description  as  required: 


RETRIEVAL  REQUEST  (EXAMPLE  -2) 


o  4 

nWI 


nc)C 
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Diite/Timc  tn:  v*  !  ^  ^  2  _ .QfMhQ. 

Diite/riinc  Required:  j  J-  j, j. _ .!. ^•'I  \.,-t  ^.C1 

Date/Time  Out: _ 


Rtqut.it  Oiiijinjlof  (Name,  Code,  Bldij/Rin.  Telephone): 

j'-‘ 


v.)  .  I  U.  O'**! f  K'|  '•*  VI  t^  ,  0  3  I  , 


VI 


Ui-2  -  -H?'l 


Airci  df  t 
Type. 


£_riQ 


Deficiency 

Identification 


Narrative 

Catetjory. 


•v  !  .  ,  I  »  .  .  _  *  J_  ^  , 

Statement  of  Retrieval  Requirement:  i'  ,ii.’  I,  J  I  ■  v -L  "  |)  i  i  t  *  -  t  S'  i  '  <T  j 

C'O  iU  ^  Vt  1 1  c ^  S hvV.  ■/ 

|  t  t  n  i  1  I  |  t  T  Csc'l  ;  K  )  *-  I  '•  i 

£l:  i  I 


}  »\I'C  )  >'iiM  I  IC  I  ; 


/•% 


c>4- 


a  , 

j  lei 


(6.1)  Vi 

C  X  *}  L  V.  *  to  I  4 1  Cj  V 4*  t  >  ,  e.i  C  'i  •-  f 


ef  I  ^  •  S  !•  t  -j  I  le-f  1  '  lw 

f  '  ;  c-  '  V  e  j' I\ 

*  **  t 

ut.'  i  l  qi  i  c'r  , 


Printout  Format: 

Deficiency  Status  Summary: 
Detailed  Deficiency  Report:. 

Statistical  Compilation: _ 

Desciiption  as  required: 


RETRIEVAL  REQUEST  (EXAMPLE  ^3) 
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tfUMHUMitia 


MiiiiilaHHi 


STATUS:  C  10/74  SERIOUSNESS:  It  RESPONSIBILITY: 


DETAILED  DEFICIENCY  REPORT  (EXAMPLE  *3) 


APPENQIX  F 
DEPARTMENT  OF  THE  NAVY 
NAVAL  AIR  SYSTEMS  COMMAND 
WASHINGTON,  D  C.  20361 


l  ; 
'  * 


O  T 


;‘iu  i 


V  *  *  ■  .*  1 

*  if? 

+  ■  *  i  * ; 


ih  m^y  «ir(i  xo 

53130:JCH 
Ser  314: 

AUG  9  37C 


From:  Commander,  Naval  Air  Systems  Command 


To:  Commander,  Naval  Air  Test  Center,  Patuxent  River,  Maryland 


Subj:  Automated  Tracking  of  F-18  Deficiencies 


COMNAVAIRTESTCEN  ltr  Ser  SA18/544  of  3  Jan  1978 
NAVAIRSYSCQM  ROT&E  FLAG  System  Volumes  I,  II  and  III  by 
John  C.  Hemingway  and  CDR  Paul  R.  Chatelier,  NAVAIRSYSCOM, 
Washington,  D.C.  of  Feb  1977 

Sample  Detailed  Deficiency  Report 
Sample  Deficiency  Report  Status  Summary 


1.  Reference  (a)  highlighted  the  need  for  an  automated  system  of  tracking 
the  status  and  disposition  of  F-18  weapon  system  deficiency  reports  and 
recommended  that  the  Naval  Air  Systems  Command  (NAVAIRSYSCOM)  F-18  Program 
Manager  (PMA-265)  employ  a  derivative  of  the  existing  NAVAIRSYSCOM  Feed¬ 
back  Loop  Action  Generation  (FLAG)  System  (described  in  reference  (b)) 
for  this  purpose.  During  the  past  several  months,  AIR-531  and  the  Naval 
Air  Test  Canter  (NAVAIRTESTCEN)  F-18  Program  Office  have  reviewed  existing 
FLAG  System  capabilities  in  the  light  of  recent  F-14  Deficiency  Report 
management  experience  in  order  to  anticipate  NAVAIRSYSCOM  and  NAVAIRTEST¬ 
CEN  F-18  program  Deficiency  Report  management  and  processing  requirements. 
Enclosures  (1)  and  (2)  contain  the  revised  system  output  formats  which  are 
expected  to  satisfy  F-18  program  Deficiency  Report  management  needs. 


2.  In  order  to  "debug"  the  revised  FLAG  system  and  have  it  in  operation 
by  the  start  of  F-18  testing  in  January  1979,  AIR-5102F,  AIR-531  and  AIR- 
5QJZ  will  work  with  the  cognizant  AIR-05  F-18  Project  Support  Officers 
(PSOs)  In  upcoming  months  to  Initialize  the  system  with  existing  F-18 
Design  Review  Request  for  Action  (RFA),  Proposed  Specification  Change 
Note  (PSCN)  transactions  and  mockup  chits.  The  tracking  of  deficiencies, 
starting  with  the  RFA's  and  continuing  throughout  the  life  cycle  of  the 
F-18  program,  will  allow  documentation  of  technical  decisions  for  each 
deficiency  and  provide  significant  Improvement  in  the  preparation,  review 
and  management  of  deficiency  reports. 

3.  The  degree  of  correlation  between  the  RFA/PSCN/mockup  chit  entries 
and  subsequent  F-18  FSO  deficiency  reports  is  expected  to  provide 
COtffiAVA I RS YSCOM  with  a  report  card  on  the  effectiveness  of  the  F-18  design 
review  process.  In  a  similar  manner,  correlations  and  discrepancies  between 


-Vr  ' 
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Subj:  Automated  Tracking  of  F-18  Deficiencies 


F— 18  FSD  Deficiency  Reports,  the  corresponding  NAVAIRSYSCOM  decisions,  and 
subsequent  fleet  Unsatisfactory  Report  submissions  will  provide  COMNAVAIR- 
SYSCOM  with  a  means  to  measure  the  effectiveness  of  the  entire  F-18  develop¬ 
ment  program,  the  impact  of  FSD  program  funding  constraints,  and  the 
decision-making  batting  average  of  key  participants.  Including  the  contractor’s 
engineering  and  management  team.  It  is  anticipated  that  this  comprehensive 
documentation  will  have  an  overall  enhancing  effect  upon  the  quality  of 
decisions  made  during  F-18  FSD,  as  well  as  future  NAVAIRSYSCOM  programs. 

4.  Upon  completion  of  FLAG  computer  program  initialization  with  F-18  RFA/ 
PSCN/mockup  chit  information,  AIR-5102F  and  AIR-531  will  provide  all 
NAVAIRSYSCOM  and  field  activity  participants  with  more  definitive  information 
on  the  use  of  the  FLAG  System.  The  decision  to  recommend  use  of  the  FLAG 
System  on  other  NAVAIRSYSCOM  weapon  system  development  programs  will  be 
made  by  AIR-510  following  review  of  the  F-18  initialization  effort.  In 
the  Interim,  suggestions  and  recommendations  to  further  Improve  and  adapt 
the  Flag  System  and  enclosures  (1)  and  (2)  to  user  needs  should  be  forwarded 
to  AIR-531  (J.  Hemingway,  AIR-53130,  AV-222-7487). 


Copy  to: 

MAVWEPENGSU  PACT 

NAVWPNCEN 

PACMISTESTCEN 

Sub-Board  of  Inspection  and  Survey 
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Attechewnc  B 
Enclosure  (1) 


CONTRACTOR  RECOM:  MAC 
ACT I ON/ (STATUS) 

OPEN  -  (4/79)  CLAIMS  THEIR  CONFIGURATION  IS  WITHIN  SPEC  (MIL-L-006730B)  WF.  CLAIMS  SWITCH 
OPTIMALLY  LOCATEO,  DEFICIENCY  TO  BE  CORRECTED  THROUGH  PILOT  USAGE. 
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ACtochaenC  B 
Enclosure  (1) 


SAMPLE 

DEFICIENCY  REPORT  STATUS  SUMMARY 


LIST  OF  REFERENCES 
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